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(57) Abstract: An electronic book selection and delivery system distributes electronic text and graphics to subscribers. The system 
contains an operations center, a video distribution system or a variety of altemadve distribution systems, a home subsystem, and a 
billing and collection system. The operations center and/or distribution points perform the functions of manipulation of text data, 
security and coding of text, cataloging of books, message center, and uplink functions. The home subsystem connects to a video dis- 
tribution system or variety of altemathre distribution systems, generates menus and stores text, and transacts through communicating 
mechanisms. A portable book-shaped viewer is used for viewing the text The viewer 
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can receive digital video data from a variety of sources, including the Internet by wired or wireless connections. The viewer may use 
the Internet connection to control and receive multiple line video feeds such as from a world watch live system. Users may be able 
to view a plurality of remote locations in real time and remotely control a video picture of a distant location. The remote control may 
be either actual control of a remote video camera or perceived remote control by the manipnlation of atufiovisnal data streams. Text, 
graphics, and other video information supplement one or more video pictures to provide an educational and entertaining system. 
Information is accessible to users who are viewing multiple video pictures. The infomiation relates and descnbes what is being 
viewed. Users who have different types of equipment, with different data rates, are able to access and use the system of the present 
invention. Users may interactively communicate with a video lecturer by asking questions and receiving answers. 
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ELECTRONIC BOOK CONNECTION TO WORLD WATCH LIVE 
Related Applications 

This i^licadonis acontinuation-inrpaitofU.S. ^plication SeiialNo. 07/991,074 
entitled TELEVISIONPROGRAM PACKAGING AND DELIVERY SYSTEM WITH 

5 MENUDRIVEN SUBSCRIBER ACCESS, filedDecember9, 1992, U.S. i^lication Serial 
No.08/336^47eiititledELECTRONIC.BOOKSELECnONANDDELIVERYSYSTEM, 
filedNovember?, 1994, U.S. Application Serial No. 08/1 60, 194 and PCT/US93/1 1606 
entifled ADVANCED SET-TOPTORMINALFORCABLETELEVISIONDELIVERY 
SYSTEMS, filed Decanber 2, 1993, U.S. Application Sedal No. 08/906,469 entitled 

10 REPROGRAMMABLETERMINALFORSUGGESTINGPROGRAMSOFFEKEDON 
ATELEVISI0NPR0aRAMDELIVERYSYSTEM,filedAugust5, 1997,U.S. AppUcation 
SerialNo. 09/191,520entitledDIGITALBROADCASTPROGRAM ORDERING, ffled 
November 13, 1998 andU.S. Application SerialNo. 08/923,091 entitled APPARATUS 
FOR VIDEO ACCESS AND CONTROL OVER COMPUTER NETWORK, 

15 INrCLUDINGIMAGECORRECTION,filedSq)tember4,1997. These iq)plications are 
incorporated by reference herem. Also incorporated by reference are co-pending U.S. 
AppUcation SerialNo. 09/237,827 entitiedELECTRONICBOOKHAVlNG LIBRARY 
CATALOG MENU AND SEARCHING FEATURES, filed January 27, 1999; U.S. 
AppUcationSerialNo. 09/237,828 entitledELECTRONICBOOKELECTRONICLINKS, 

20 filed January 27,- 1999; U.S. AppUcation Serial No. 09/391,461 entitled VIDEO 
CONFERENCING WITH ANELECTROMCBOOK VIEWER, filedSeptemberS, 1999; 
and U.S. AppUcation Serial No. 09/289,956, entitled ELECTRONIC BOOK 
ALTERNATIVE DELIVERY METHODS, filed April 13, 1999. 
Field Of The Invention 

25 Hiis invendonrelates to &e distribution of audiovisual signals through commuaications 

networks such as computer networics and servers and their subsequ^t storage in and display 
onan electronic bookreador. Theinventionhasparticularusewithre^ecttoglobalnetworics 
such as the Internet and "World Wide Web". The invration also relates to education. 
Particularly, the invention provides an alternative to in-person classroom instruction. 
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Description Of Related Art 

There is a constant desire to improve education and knowledge at all levels. It is 
thoughttiiatlrue human progress can only be achievedif people's understanding of each other 
is improved and if people's understanding of nature and the environmrat is improved. 
S Traditionally, education and knowledge have been obtained in schools from classroom 
instmction and from the reading of books. 

The disadvantage of cun:entclassroom instnictional systems is that st^ 
physically present in the classroom to participate in tiie educational process. Therefore, 
students who are geographically displaced from the location of the classroom often cannot 
10 attend class instmction as often or as timely as students who are nearby to the 
classroom. 

The disadvantage of textbooks is that they are often not kept cunent with recent events 
or technological changes. Textbooks are usually only updated on ayearly or less frequent 
basis, while iitq>ortaQt changes may occur monthly or more frequently. Also, to save fimds, 

1 5 schools may not purchase new textbooks even though the textbooks have been updated. 
Therefore, the new knowledge, although available, is not communicated to students. 

Recently, audiovisual presentations have begun to be used in the field of education. 
These systemsmay provideplayback of arecording of a lecturer who provides apresentation 
on an educational topic. For example, students may learn about math from watching a 

20 videotape ortelevisionbroadcast of amafh professor's lecture. Education can also occur on 
a more informal basis. For example, specialty channels in the United States such as the 
Discovery Chamiel® and The Learning Channel® (headquartered in B ethesda, Maryland, 
U.S.A.) broadcast educational programming that both entertains and educates a diverse 
viewership. 

25 Thedisadvaotageoftheseaudiovisualsystemsisthattheyarenotinia:^ Students 

areunable to ask questions, and the lecturer is unable to tailorthe presentation ofmaterial to 
the specific needs ofthe cunent student audience. Consequenfiy, the needs oftfae students are 
not met 
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Cable andbroadcast television are conm^ 
tolaigenumbersofviewersequippedwifhreceiveiskn^ Byreceiving 
abroadcast, cablecast or satellite signal, users are able to view scenes ftom remote locations 
and observe newsworthy events which occur fiar from the user's location. However, 
conventioiial television isaone-waymedia in wMch users cannot communicate \^ 
or die broadcaster. 

Recently, the advent of the Internet, andfhe Worldwide Web, in conjunction with the 
proliforation of personal computers, has allowedpeople to exchange information and ideas on 
a global and inexpensive basis. Generally speaking, &e Mtemet is a large computernetwork 
which connects host conq)uters. Users with a computer, modem and telq>hone line commonly 
call via telephone to connect with a host The host, being in communication with other hosts 
(connected to ofherusers)is able to transferinformation between users. The Internet is used, 
for ^cample, to transfer, data files, still images, sounds andmessages between virtually any two 
points in the world with telephone access. 

The use of &e Internet has increased dramatically since 1981, when proximately 300 
host conq)uters were linked together. In 1 989, the number oflinked host computers was fewer 
than 90,000; but by 1993, over a miUion host computers were connected. Currently over 10 
million host conq)uters are linked (not including thepersonal conputers people use to access 
these hosts via modems) and as many as 40 million people around the world may have access 
tothelntemetmedium. Thisnumber is expected to growto 200millionby theendof 1999. 

Users on the Internet are able to transfer text, grs^hics, and still pictures between 
remote locations. OtiiertypesofinforoiationwUc^canbe transmitted include files conta 
prerecorded sequences of images. To view these images, uscts download a large data file, and 
afterrunning appropri ate software, see a sequence of images on the computer screen. These 
images typical^ are not provided in real time, and are not viewable while the user is accessing 
the Internet 

Therefore, even though the Internet isatwo-waycommunicationmedium, it is not 
currenflyberogutilizedtoprovidevideoinformationandaudiovisualpresm Thisisa 
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disadvantage, in that a large number of people have been accustomed to television audiovisual 
presentations, and prefer an audio-video presentation to a textual or graphical 
presentatioiL 
Summary Of Invention 

The electronic book selection and delivery system is a new way to distribute books to 
bookstores, public libraries, schools and consumers. The technological breakthroughs of this 
invention provide a secure electronic syst^ jfor both delivering selectedbooks and receiving 
payments. Thesystemhasanunusual combination offeaturesthatprovidesfh^ 
a daily use household appliance that has a high tech aura while being very practical, portable, 
and easy to use. 

An advantage of the system is that it eliminates the distribution of any physical obj ect 
such as a paper book or computer memory device from any book or text distribution system. 
The purchase of a book becomes a P AY-PER-READ™ event avoiding the overhead, 
'^ddle-men," printing costs, and time delay associated with the current book distribution 
systOTi. Published material and text such as the President' s speech, a new law, a court decision 
on abortion, or O.J. Simpson's testimony can be made immediately available to the consumer 
at anominal fee. Alternatively, books may be made available fiee to the end use consumer, 
subsidized by advertisers who sponsor books or embed advertising within the books. 

The system is anovel combinationof new technology involving the television, cable, 
telephone, and computer industries. It utilities high bandwidth data transmissions, strong 
security measures, sophisticated digital switching, hig^resoMon visual dispkys, novel cch^ 
and user fidendly interface software. 

The primary components of the text deUvery system are the subsystem for preparing 
the text for delivery and the subsystem for receiving and selecting text that was delivered. An 
embodiment of the system includes additional components and optional features diat enhance 
thesyst^ The syst^niay be configured for use by bookstores, pubUcUbraries,scho^ 
consumers. Iaoneembodiment,thesystemforcoiisumeruseisniadeiq)offoursubsysteins, 
namely: (1) an operations center, (2) a distribution system, (3) ahome subsystem including 
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reception, selection, viewing, transacting and transmission capabilities, and (4) abilling and 
collection system. Alternative configurations of tiie system are defined to allow for a variety 
of traditional and non-traditional delivery methods. 

The operations center performs several primary fimctions: manipulating text data 

5 (including receiving, formatting and storing of text data), security encoding of text, cataloging 
ofbooks, providing amessaging coiter capability, and performing uplinkiunctions. In one 
embodiment, the system delivers the text from the operatioiis ceriter to consumer homes by 
insertingtextdatawitfaih analog video sijgnals. Ilieiri^ertionoftextisgeQenillyperfonnedwith 
an encoder at an uplink site that is within ornear the operations center. The system can use 

1 0 several lines of flie Vertical Blanking Interval (VBI), all the lines of the analog video signal, a 
digital video signal or unused portions ofbandwidth to transmit text data Using the VBI 
delivery method, the top tenor twenty booktities may be transmitted with video duringnormal 
programming utilizuig existmg cable or broadcast transmission capability without disruption to 
the subscriber's video reception. Using the entire video signal, thousands ofbooks maybe 

15 transmitted within just one hour of air time. Nearly any analog or digital video or data 
distribution system may be used to dehver the text data. The text data may also be transmitted 
over other low and high speed signal paths including a telephone network (e.g., a public 
switched telephone network) having a high speed connection such as a digital subscriber line 
(DSL) connection and the Internet, for example. 

20 Thehomesubsystemperforms at least four fimctions: connecting to the distribution 

sy5te[n,selectingtext,storingtext,aiidtransactingflTOughacommum The 
components of the home subsystem may be configured in a variety ofhardware configurations. 
Each fimctionmay be performed by a s&panAe cong)onent, ttie components m^ be integrated, 
or the capability of existing cable set top converter boxes, computers, and televisions may be 

25 utilized. A connector, library unit and viewer unit may be used In one embodiment, tiie 
connector portion of the home subsystem receives an analog video signal and strips or extracts 
the text from the video. The home library stores the text signal, provides a user fiiendly 
softwareinterfacetotfaesystemandprocesseslhetransactionsatttieconsumerhome. The 
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viewer provides a screen for viewing or menus and novel user friendly controls. 
Alternative embodiments are presented that support delivery of text using a variety of 
communication mechanisms. 

The viewing device may be aportable book shaped viewer which stores one or more 

S booksforviewingandprovidesascreenforinteractingwiththehomeUbraryunit. Ahigh 
resoludondisplay is used to both read thebooks and to interact wilhthe home library software. 
Ihone embodiment, an optional phone cormectororretum-pathcaU 
telephone calls and, with the aid of the library, transmits the necessary data to complete the 
ordering and billing portion of the consumer transaction. Alternative embodiments are 

10 presCTitedthatsupportorderingandbillingusingavaridyofcoinmunicationmecl^ The 
user fdoidly controls include abookmark, current book andpagetumbuttotL The billing and 
collection s^^stemperforms transactionmanagement, authorizations, collections andpublisher 
payments automatically. 

A system sunilar to the system for consumer use maybe usedinbookstores, schools 

1 5 and public Ubraries. 

In one aspect of the invention, video is collected at aremote site. The' term * Video," 
as usedherein, includes stereophonic ormonophonic audio signals which may accompany a 
video signal. Additionally, 'Video" is usedbroadly herein to include still images, groups of 
related still images, animation, graphics, pictures, or other visual data, including textual data. 

20 The remote video information may be obtained fiom a video cassette, CD ROMs, television 
channels, one or more video cameras, or other well known sources. If video cameras are 
used, tiiey may be connected to a computer so that they are remotely controllable, or they may 
be oriented such that a perception of control can be created for users. The video may relate 
to remote sites of interest, such as a pyramid in Egypt, or the images may relate to an 

25 educational lecture being conducted at a remote site. 

The collected video may be transferred to a web site, either in compressed or 
unconq)ressedfoim. ThevideoixiaybephysicaIlytcan^)artedorniaybetransmittedthroug^ 
a communications medium to the web site. 
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The web site may contain a storage media that stores some or all of the video. 
Additionally, the web site may pass camera control commands, if £5)plicable, to ttie remotely 
controlled cameras or may simulate the remote control of a camera. One function of the web 
site is to pass video to a plurality of users, trough a communication media such as the Internet, 
in response to user selections. The video passed to the plurality of users may be live video 
being fed to the web site, ormaybe stored video. A number of video servers may be used to 
outputthevideotothe\isersthroughthecommuiucationsmedia,suchastheM The 
video may be tailored by the web site for the particular user's hardware, including data 
communication equipment, or memory size, i.e. the datarate matches the highest speed which 
the user's equipment can handle. 

Usersreceiveand displaythe video sentfrom the web site. Many simultaneous video 
pictures may be received. The quality and frame rate of the video is dependent on the user's 
communications hardware. Users having electronic book viewers with hi^-speedmodems 
or cable modems may receive higher quality video. The users are able to send conmiands 
and/or queries to the web site. The conmiands and queries are forwarded to remote locations 
to control remote cameras or query remotely located instmctors. Alternatively, the commands 
may cause the web site to change from among many video signals with different camera angles 
or locations (or to transmit a different portion of a wide angle image), causing the user to have 
a perception of remote camera control. The user's commands may also cause a different 
portion of a received wide angle image to be displayed, giving the user aperception of camera 
control. 

Inadditionto video, the web siteprovides information, suchasgnphics and text, which 
may be related to the video or may be a separate program. This information may be 
automatically s\q)phed,orprovidediq>onuserrequest Therefore, the user may be provided 
witha coD:^)rehensivesetofinforniationconcemingr5motesites,remotely loca^ 
or oth^ information or data, enabling tiie user to be quickly educated aboutthe remote site of 
interest 

Brief Description Of Drawings 
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Figure laisablockdiagramof theprimaiy (X>n]^ 
and delivery system. 

Figure lb is ablock diagram of an electronic book selection and delivery system that 
uses a composite video signal. 
5 Figure 2 is a schematic showing an overview of the electronic book selection and 

delivery system. 

Figure 3 is aschematic of a dehvery planforttie el!Wtconicbookselectionaiid(^^ 
system. 

Figure 4 is ablock diagram of the operations center. 
10 Figure 5a is a flow diagram of processing at the operations center and uplink. 

Figure 5b is a block diagram of a hardware configuration for an uplink site. 
Figure 6a is a block diagram of a hardware configuration for a four component home 
subsystem. 

Figure 6b is a schematic of a two unit home subsystem. 
15 Figure 7 is a flow diagram of processes performed by a video connector. 

Figure 8 is a block diagram for an example of a library unit. 
Figure 9 is a flow diagram of processes performed by a library lanit on the received 
data stream. 

Figure 1 0 is a flow diagram of processes performed by a library unit on infonnation 
20 requests fix>m a viewer. 

Figure 11 is a block diagram showing the componmts for an example of a 

viewer. 

Figure 1 2 is a flow diagram of processes performed by a viewer on an information 
request fi'om a subscriber. 
25 Figure 13 is a chart depicting amenu structure and sequencing of menus in amenu 

system. 

Figure 14a is a schematic of an intzDductoiy menu. 

Figure 14b is a schematic showing an example of a main menu. 
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Figures 14c, 14d, 14e, 14^ 14g, 14h, 14i and 14j are schematics showing examples 
of submenus. 

Figure 15 is a schematic diagram of an electronic book system for a bookstore or 
public library. 

S Figure 1 6a andFigure 1 6b are schematics ofhardware modifications or upgrades to 

a set top converter. 

Figure 1 7 is a schematic showing a set top terminal that includes a data receiver and 
data transmitter. 

Figure 1 8a is a schematic of a book-on-demand system. 
1 0 Figure 1 8b is a schematic of an operations center supporting a book-on-demand 

system. 

Figure 19 isa(&agramdq)ictingcomponeatsusedfor delivery of electronic books ovar 
the Internet. 

Figure 20 is a schematic of a page of an electronic book having electronic links; 
1 S Figure 21 is a schematic of a page of an electronic book with the electronic links 

shown; 

Figure 22 is a schematic of a show links submenu; 

Figure 23 is an exanq)le of links in an electronic book; 

Figure 24 is a block diagram of an embodiment of &e invention where remote video 
20 is provided to a web server by videocassette and by ordinary television. 

Figure 25 is ablock diagram of an embodimmt of the invention where rmiote video 
is provided by remotely located cameras and a communicationnetwoxk carries the video to 
the web server. 

Figures 26a and 26b are ablock diagrams of an embodim^ of the inventionusing the 
25 embodiments of Figures 24 and 25 with remotely controllable cameras. 

Figure 27 shows remote cameras positioned around abuilding for perceived camera 
control. 

Figures 28a - 28B show video images firom specific cameras shown in Figure 27. 
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Figure 29 shows remote cameras deployed to follow a parade route. 
Figures 30a and 30b show remotely controlled cameras at a remote location. 
Figures 31a and 3 lb show a single remote camera at a remote location, where the 
camera has a 180 degree spherical (or other wide angle) lens. 
S Figures 32a and 32b are block diagrams of a server platform. 

Figure 33 is ablock diagram of communications paths from tiiie server site to remote 

users. 

Figure 34 shows a home page in accordance with an embodiment of the 
invention. 

10 Figure 35 shows a "society" page in accordance with another embodiment of the 

invention. 

Figure 36 shows a **map" page of remote camera locations throughout the 

world- 
Figure 37 shows a 'Svatch" page containing live video feeds fiom five remote cameras. 
IS Figure 38 shows a page directed to determining tiie user's data rate. 

Figure 39 shows a page of an interactive lecture. 

Figures 40 and 4 1 show pages of an embodiment of the invention that combines live 
video, prestored video, graphics, and interactive questions. 

Figure 42 shows a flow diagram of a method of automatically monitoring and panning 
20 an area using perceived camera control. 

Figure43 isanexemplaiyscreendisplayofthepFeseQtinvendon,showingvideoand 
also showing video data. 
Detailed Description Of Invention 

Figure 1 a shows an electronic book distribution system 1 00 that may be used for 
25 distributing an electronic book. Acontentprovider 1 lOmaypublishhardcopy versionsof 
books or other printed media including newsps^ers, magazines, and product catalogs, for 
example. Thecont^pn^viderllOmayconv^printedinatmalstoanelectrom^ 
provide the electronic fonnattedmaterials to adistribution point, or c^ter 120, overuplink 
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pathllS. TheuplinkpathllSmaybeawiredorawirelesspath. Theiq)liiikpathll5may 
be a telecommimications network, for example. Theuplinkpath 1 15 maybe a satellite relay 
path or a wireless telephone path. The \q)link path 115 may involve providing electronic books 
to the distribution center on a fixed media, such as a CD-ROM, for example. 

5 InFigure la, the content provider 1 10 and the distribution center 120 are shown as 

separate components of the electronic book distribution system 1 00. However, &e content 
. provider 110 and the distribution ceiiter 120 may be.co-located. 

The distribution center 120 may convert printed matter into an electronic format. 
Alternately, the distribution center 1 20 may receive electronic files fix)m an outside source, such 

10 as the content provider 110. The distribution center 1 20 may process and store electronic 
books. 

The distribution center 120 distributes electronic books. Thedistributionmaybe, for 
example, over distributionpath 125, distributionnetwork 130, and distribution path 135 to an 
electronicbooksubsystem or tOTninal 140, whichmay include an electronicto^ 

15 shown). The terminal inay also be a television, a set top ternainal, apersonal computer, or 
similar device. An apparatus and method for distributing electronic books is disclosed in 
greater detail later. The distributionnetwork 130maybean electronic bookstore, anintemet 
web site, a wired or wireless telecommunications netwodc, anintianet, aradioprogram delivery 
s>^tem, a television program delivery system, including cable television, satellite television 

20 broadcast, and over-the-air broadcast, for example. The electronic book distributionnetworic 
130 could include direct delivery through a mail delivay system of electronic books on a fixed 
media, such as a CD-ROM, for example. 

Figure lb shows components of an electronic book distribution system 170 using a 
television program delivery system to distribute electronic books. 

25 In the embodiment shown in Figure lb, the components of the electronic book 

selection and delivery system 170 are an encoder 174, a video distribution system 178, a 
coimector 182, and a text selector 1 86. The encoder 174 places textual data on a video signal 
to form aconq>osite video signal AUhou^theconq)ositesignalmaycontainonlytextualdata, 
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itusuaUycamesbothvideoandte^ctuald^^ Avarietyofeqmpmentandmetfaodsniaybeiised 
to encode text data onto a video signal. The video distribution system 178 distributes the 
composite video signal firom the single point of the encoder 174 to multiple locations, which 
have connectors 1 82. The connector 182 receives the digital or analog video signal from the 

S video distribution system 178 and separates, strips or eictracts the text data from the con:q)osite 
videosignaL Ifnecessary,theextractedtextdataisconvertedintoadigitalbitstream. Thetext 
selector 1 86 wodcs in connection with flie connector 1 82 to select text. 

Using the connector 182 and tex:t selector 186 combination, various methods of 
selecting and retrieving desired text from a composite or video signal are possible. Text may 

10 be preselected, selected as received or selected afterbeing received and stored. One method 
is for the connector 1 82 to strip or extract all the text from the video signal and have the text 
selector 186sa:eenallthetextasreceivedfiomtheconnector 182. Thetext selector 186 only 
stores text in long t^m or p ermanent memory if the text passes a screening process described 
below. 

1 5 Figure 2 shows another embodiment of an electronic book selection and delivery 

system200. The delivery system 200 includes: an operations center 250 including an uplink 
site 254, a video distribution system 208, an electronic book device, or home system 25 8 
including a video connector 2 1 2, a library 262, a viewer 266, and a phone coimector 270, 
telephone system 274, anintemetweb site 279 andabillingandcollectionsystem278. Also 

20 as shown in Figure 2, the home system 25 8 may include connections to a television 259 and 
apersonal computer 26 1 may be used to display menu screens, electronic books, electronic 
files, or any other information associated with the electronic book delivery system 200. In 
addition, the television 259 and thepersonal conq)uter 261 may provide control functions that 
replicate and supplement those of the viewer 266. 

25 TheoperationscCTter 250rBceives textualmaterial from outside sources 282 suchas 

publishers, newspq)ers, and on-line services. Alternately, the outside sources may maintain 
electronic books at the Intemet web site 279. The outside sources 282 may convert textual 
and graphical material to digital format, ormaycontractwithano&ervendorto provide this 
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service. The operations center 250 may receive the tactual and graphical material in various 
digital formats and may convert the textual material to a standard contipressed format for 
storage. In so doing, the operations center 25 0 may create apool of textual material that is 
available to be delivered to the home system 258. The textual material may be grouped by 
books or titles for easy access. 

As used herein, '"book" means textual, graphical or video information. Such 
information may be contained in any novels, encyclopedias, articles, magames, newspsq)^, 
catalogues, periodicals, or manuals or any portion or section of the above. The information 
may also be provided as a video clip, for example. The term "title" may represent the actual 
title assigned to abook, a video program, or any otha: designation indicating aparticular group, 
portion, or category of textual information. The title may refer to a series of related textual, 
video or gr2?>hical infonnation, a grouping of textual, video or graphical information, or a 
portion oftextual, video or graphical infonnation. Forexan^)le,'l.atestHailequinRomance'', 
TourC3iildReadingBooks(Ages 10-12V"*EncyclopediaBmA^fl^CA'™;«Tresito^^ 
Speech,"'TnstractionManual,""Scheduleof4thofJulyEvents,"'TetHffi 
Wade," and 'The Joy of Cooking," are suitable tities. Also, the titie maybe a graphical symbol 
oricon. Thus, apictureofawrenchmaybeatifle for arepair book, apicture of a computer 
atiUe for a computer book, a graphical symbol of a telephone atitle for a telephone book, a 
drawing of a dagger a title for a mystery book, a picture of a bat and ball a titie for a sports 
book, and a picture of tickertape a title for a business book. 

The operations center 250 includes the uplink site 254 for placing the text or other 
infonnation onto atelecommunications signal and sending the telecommunications signal into 
a distribution system. The uplink site 254 may include an encoder 204 (not shown in Figure 
2) to encode the text onto the telecommunications signal. 

Many analog and digital video distribution systems maybe used witiiihe electronic 
book delivery system 200, such as cable television distribution systems, broadcast television 
distribution systOTis, video distributed over telephone systems, direct satellite broadcast 
distribution systems, and other wire and wireless video distribution systmis. Nearly any 
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distnbutiaa system which can deliver a teLecommunications signal, inchiding a video signal, wiH 
work with the electronic book delivery system 200. It is also possible to distribute the 
electronic book without using a telecommunications signal as described in the embodiments 
presented in Section VH below, 
S The home system 258 peifom^s five fimctions: (1) connecting with a video distribution 

system; (2) selecting data; (3) storing data; (4) displaying data; and (5) handling transactions. 
. An important optional function of the home system 258 is commimicating using, in one 
embodiment, a telephone communication system 274. The home system 258 may bemade up 
of four parts: a video connector 2 1 2 or similar type of cormector for connecting with the 

1 0 distribution system 208, a library 262 for storing and processing, a viewer 266 for viewing 
menus and text and atelq)hone connector 270 for connecting witha telephone communications 
system 274. Additional embodiments are presented in Section Vn ttiat address alternative 
communication mechanisms. 

The billing and collection system 278 maybe co-located with the operations c^xter 250 

15 or located remote bom the operations center 250. The billing and collection system 278 may 
be in communication with the home system 25 8 using telephone-type communication systems 
(for example 274). Any of a number of communication systems as presented in Section VII, 
such as acellular system or tibe Internet, will operate with thebilling and collection syst^ 278. 
The billing and collection syst^ 278 records the electronic books or video, orportions of t^ 

20 or video that are selected or ordered by the subscriber. The collection system will charge a 
subscriber's credit account or bill &e subscriber. In addition, the billing and collection system 
278 may monitor that amount due to publishers or other outside sources 282 who have 
provided textual data or oth^ services such as air time to enable the text delivery system 200 
to operate. 

25 AIsoshowninFigure2isanintranet279'. Theintranet279'maybeusedasapart 

ofaprhratedistributionnetwo±fodistributirigandciiculati^ Forexample, 
a university Hhrary may use ttie intranet 279 ' to diculate electronic books to univ^ty students 
and professors. 
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Figure 3 is an expanded overview of a delivery plan 301 for the electronic book 
delivery system 200. ItisacomprehensivedeliveryplanSOl to support various lypes of users 
and various billing systems* Figure 3 shows thatpublishers 282 may provide text transfer 302 
to theoperations center 250' and receive payments 306 from the billing and collection systOTi 

5 ' 278\ Aseparatechanneluplii)ksite254' isshowninthiscoiifigurationreceivingdataSlO 
fiom Ifae operations center 250' . The operations center 250' has fliree separate sections (3 1 8, 
322, 326) one for text recdving, formatting and re-entry 3 
322 and a third section for catalog and messaging center functions 326. 

The collection and billing system 278 ' shown has two sections (330, 334) one for 

10 transaction management, authorizations and publisher payments 330, and the other for 
customer service 334. The customer service section 334 provides for data entry and access 
tocustomeraccountinfixrmatioa Transaction accountinginfonnation338 is supplied to credit 
card companies 342by the transaction management section 330 of the billing and collection 
system 278'. The credit card companies 342 provide billing 346 to customers eitiier 

1 5 electronically or by mail. 

Methods for communicating between the subscriber base 348 and the billing and 
collection system 278 ' include: by telephone switching 350 alone, cellular switching 354 and 
telephone switching 350 combined, and by use of the cable system 358 and the telephone 
switching 350. The system shown supports both one-way 362 and two-way cable 

20 communication 366 with subscribers. Additional communicationmethods arepresented in 
Section Vn. PubUcUbraries and schools 370 as weU as bookstores 374 niay use the delivery 
system 301. 

PubUc libraries and schools 370 could have a modified system to allow the viewer to 
be checked-out or borrowed while bookstores 374 would rent or sell the viewer and sell 
25 electronicbookdata. The bookstores 374 as well as the public libraries and schools 370 may 
be serviced by cable 378. Optional direct broadcast systems (DBS) 382 can also be used 
with the system 200. 
1. The Op^ations Center 
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Figure 4 is a schematic of the operations center 250, which includes the uplink 254. 
The operations center 250 may gather text orhooks by receiving, formatting, storing, and 
encoding. A data stream 302 contaiiiingtextmayhereceivedattheoperationscenter250by 
a data receiver 402. The data receiver 402 is under the control of a processor 404. After 
5 recqption, flie datastreamis fonnattedusing digital logic for formatting 406 wbidiis also under 
the control of the processor 404. If any additional text is generated at the operations center 
250 locally for ios^oninto the distributed.signal, the text generation is handledthrough text 
generator hardware 410, which may include a data receiver and a keyboard (not shown). 
Following processing by the text generator 4 1 0, the additional text can be added to the text 
10 received by the combining hardware 414 that includes digital logic circuitry (not 
shown). 

The processing at the operations center 250 is controlledby aprocessor 404, which 
uses an instruction memory 416. The processor 404 andinstructionmemory416maybe 
siqypUedby apeisonal computer or noini-computer, for example. To perfomi the catalog and 

15 messaging functions, the operations center250uses acatalog and message memory 420 and 
the text generator 410 if necessary. 

The data stream of text, catalog and messages may be encoded by security module 
encoding 424 prior to being sent to the iq)link module 254. Various encodmg techniques may 
beusedby the security encodmgmodule424 such as the commercial derivative ofNS As 

20 enotyption algorithm (DataEnayptionSystem(DES)) and Gaerallnstium^^^ 

Following encoding, the encodedtextmay be stored in textmemory 428 priorto being sent 
to the iq)link 254. A first-in-first-out text memory arrangraient may be used under the control 
of the processor 404. Various types of memory may be used for the text memory 428 
includingRAM. Theoperationscenter250mayusefiO[eserverte(^ologyfortfaete!ctmem^ 

25 428tDcatalogandspoolbooksfortransinissiona5isdescribedbelow. The opea:ations center 
250 may also store the electronic book as compressed data files. 

hi an snbodiment, to transmit textual data, the distribution system 208 (see Figure 2) 
may use high bandwidth transmission techniques such as those defined by the Nortii American 
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Broadcast Teletext Standard (NABTS) and the World System Teletext (WST) standard. 
Using the WST format (where each line of tiie Vertical Blanking Interval contains 266 data 
bits), a four hundred page book, for example, may be transmitted during regular television 
programming using four lines of the Vertical Blanking Interval at arate of approximately one 
. book every 1 .6 minutes (63,840 bits per second). Alternatively, books maybe transmitted 
over adedicated channel, whichintemq)tsprogi:amming so that 246 lines ofvideo canbe used 
to transmit approximately 2,250 books every hour (3 .9 Mbits per second), A teletext type 
format is the simplest but possibly the slowest text format to use with the electronic book 
delivery system 200. In either event, an encoder 204 may be used at an uplink site 254 to 
insert textual data into the analog video si gnal. In many other respects, the delivery of the 
textual infonnationmay be completed using an existing cable television plant and equipment 
Alt^native transmit formats and delivery systems are presented in Section Vn. 

Figure 5a is a flowchart of steps involved in processing text from the publisher or 
provider 282 that may occur at the operations center 250. As shown in block 500, the 
publisher 282 processes data files of text for books, compresses, mcrypts and sends the data 
files to the operations center250 oruplink254. Text files for books may be sent one book 
at a time. As shown in block 504, the uplink 254 or operations center 250 receives and 
processes the data stream from the publisher 282. Generally, part of ttiis processing includes 
encryption and error correction. Text files may be delivered for receipt by multiple home 
subsystems simultaneously, or to a specific individual home subsystem. 

In Figure 5a, the electronic books are distributed to consumers using a video 
distribution system such as a cable television system. However, the electronic books may also 
bepackaged as datapackets and distributed over other telecommunications netv^oiks such as 
a digital wireless telephone network, for example. 

Li one embodiment, as shown in block 508, files are broken into smaller packets of 
informafioiL Headerinformationisaddedtothepackets. Thebitstreamisconvertedfix>ma 
serial digitalbit stream to an analogbit stream that is compatible with anNTSC video signal 
Block 512 shows the switching of analog data into ftie video lines of a video signal. The analog 
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data may be placed either in the VBI or the active video lines. In some instances, unused 
portions of bandwidth (such as 5-40 MHZ, 70-75 MHZ, 100-109 MHZ or other guard 
bands) may be used instead ofthe video lines. Alternate transmissionmethods are presented 
in Section Vn. 

S Figure 5b is an exanq)le of a hardware configuration to perform some ofthe functions 

for blocks 508 and 5 12. A video feed 5 1 6 is received and processed through a sync stripper 
. 520. The stripped sync signal 532 is used by the digital logic control 524. The digital logic 
control 524 receives the sync signal 532 and a serial digital bit stream 528 for processing. The 
digital logic control 524 passes the serial digital bit stream to the Digital to Analog converter 
10 536andoutputsacontrolsignal540forthevideoswitch544. The video switch 544 integrates 
the video feed 516 and analog data stream 548 into a video feed with analog data signal 
inserted 552, 

As an alternative to cable, broadcast or other television delivery methods, the public 

telephonesystemmaybeusedtotransmitbookstothesubscribers. An average book would 
15 takeabout7irrinutestotransrQitoverthepublictelq)honesysterQ. Using the telephone system, 

it is notnecessaiy to combine video and text into acomposite signal. Inmost other respects, 

the operations center would remain similar whether text delivery was by telephone or cable. 

File server technology (such as that described in U.S. Patent No. 5,262,875, entitled 

AUDIOAODEOFII^SERVERINCXIIDINGDECOMPR^ 
20 et al., and, U.S. PatentNo. 5,218,695, entitled FILE SERVER SYSTEMHAVINGfflGH-SPEED 

WRITE EXECUTION , issued to Noveck, et al., incorporated herein by reference) may be used 

at the operation cent^ with a telephone system text delivery method. 

As another alternative to cable, television, and telephone system delivery, the public 

telephonesysteminaybeusedtoprovideaccess to the Internet, where the Intemetweb site 
25 279maybe accessed. Electronic books may beordea?ed,paidfor, and dehvered directly fiom 

the Internet web site 279 over the telephone system. 

When a wireless telephone network is used to distribxite electronic books, or otherwise 

communicate with the home system 258, die home system may receive data using anyone or 
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more standaidpiolocols including time divisioninultiple access (TDMA), code divisionmultiple 
access (CDMA), Global Systems for Mobile Communications (GSM) and Advanced Mobile 
Telq>hone System (AMPS) protocols. 

laany ddiveiy systemusing the telephone syston, indiv^ 

S Aeelecttomcbookdeliv^rateby incoiporatinghi^speedmodons or other communications 
devices such as an Integrated Services Digital Network (ISDN) connector, or by use of a 
Digital Subscriber Line (DSL). . . 
n. The Home Subsystem 

The hardware configuration for a four componrat home system 25 8 is shown in Figure 

10 6a. Figure 6b shows a hardware configuration for a two component home subsystem. The 
home system 258 pafonns several functions^ such as receiving data and video transmissions, 
stripping (or extracting) the data fiom the video signal, soeening and storing the data, providing 
user fiiendly interfitce controls and software, displaying menus and text, processing 
transactions, initiating telephone calls and transmitting bilUng data. Various hardware 

15 configurations maybe utihzed to achieve the desired functions ofthe home system 258. For 
example, as shown in figure 6b, the home system 258 can be configured to utilize the reception 
and channel tuning cspdbHity of the current installed subscriber base of cable converter boxes 
and televisions 601 andnetwotked computers. The home ^stem 25 8 can also be designed 
as an advanced set top terminal converter box with menu generation cs^ability, electronic 

20 memoryandatelephonemodemasdescribedinsectionVbelow. Alternatively, the home 
system 25 8 can be configured to support alternate dehvesy and ordering methods as described 
in Section VIL 

The electronic components, whichmake up the home system 258 canbe arranged in 
a variety of ways. In (he four unit subsystem of figure 6a the viewer 266 and library 262 are 
25 whed together while fiie remaining conq)onents communicate throu^ RF transceivers 604. 
In asinq)le vezsionof Qiehome systm258 tihope are only two units, alibrary 262 and a viewer 
266. Figure 6b shows a two unit home system 258 with certain optional features. 
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The viewer 266 is generaUy eqiiippedwithaMghresolutionview^ digital 
logic (including akey 605, security 606, and a microprocessor 621 ), video graphics control 
and memory 607, power supply circuitry 602 (not shown), an optional battery 603 and an 
optional RF transceiver 604. In a two umt arrangement, the library 262 contains the connector 
function to the electronic book distribution system 208, connector function to a public 
telephone communications system, and memory 600 (whichmay be removable and portable 
6000r More specifically, the library 262 would include data stripping functions 61 7, digital 
logic 609, memory storage 600, power circuitry 610, optional connections 61 1 (including 
cellular or POJ 6110, optional battery (not shown), optional tuner module 6 1 3 and an optional 
RF transceiver 604. The connector 212 and the pubUc telephone system connection 270, as 
well as the removable portable memory unit 600 of the library 262 may be broken out into 
separate components. (Figure 6b showsaremovableportable hard diskmemory 600' with 
removable cartridges 6 1 4.) Finally, the home system 258 may include an attached keyboard 

267 or a wireless keyboard 268 . Both the attached keyboard 267 and the wireless keyboard 

268 maybe used to communicate with the viewer 266 (not shown) or the library unit 262. The 
wireless keyboard 268 may communicate usiag radio fiequency (RF) signaling, for example. 

In an alternate arrangement, all functions of the home system 258 may be incorporated 
into a single unit The fimctionsof the library 262, for example, may be carried out by a card 
or chipset in the viewer 266. All the communications devices needed to couple the home 
system 25 8 to various telecommunications networks may also be incorporated into the viewer. 
All interfaces between ttie home system 25 8 and the subscriber may be included with the 
viewer 266. In this embodiment, the viewer 266 may include a communication device for 
receiving inputs fix)m a sq)arate keyboard The viewer 266 may also include abuilt-in video 
camm608" tiiat may be used to transmit images of the subscriber. Using the transceiv^ 608, 
the camera 608" and the speaker/microphone 608', the subscriber may use the viewer 266 
for video conferencing, for sample. 

Therefore, the home system 258 may have as many as five sq)arate componCTts, 
which communicate with each other. 'Ilietwo,fliree,fourorfivesq>aratecanq)onentswhich 
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make vp the home subsystem can commumcate with each otber in a variety of ways, including 
hardwired connection 615, RF transceiver 604 and other wireless methods. 

RF communications may be used in the home, allowing separate components to be 
located throughout the home without restriction. The data communicatedbetween the units 
maybe secure data. Inaddition,fhehl>rary262mayprovidepowertotheviewer266lhrou^ 
the hard wire communication link 615. 

To receive and strip data &om ayideo signal at the consumer's home, adevice such 
as a cable interface device or cable connector 212 is used. The cable connector device 
includes a tuner 613, while the cable interfece device makes use of existing tuning equipmmt 
in the home. In either configuration, data is stripped from the video signal and stored at the 
subscribers location intheUbrary 262. The phone connector 270, optional connector 611, 
and modular connector 701 initiate communications and transmit ordering and billing 
information to the operations center 250 or billing and collection system 278. A digital 
connector 619 is provided to commumcate digital infomiation with the set top 601. The Ubrary 
262 is the intelligent component of the home subsystem, incorporating the hardware and 
software necessary to store the text data, generate menus and effect the purchase transactions. 
In addition to an RF transceiver 604, the home library 262 also includes the necessary j acks 
and connections to allow the system to be connected to the viewer 266. As shown in Figure 
6b , the library 262 communicates the text data to the viewer 266 in a secure fomiat, which 
requires a key 605 for decryption. The text may be decrypted page by page just before 
viewing. 

a. The Video Connector 

Figure7 showstheflowoftheprocessesperfonnedbythevideoconnector212. TTie 
video connector 212 receives flie video signal 608, tunes to the channel containing the text data 
612, strip s the text data from the video signal 616, and communicates the text data stream to 
logic components in ttie library 620. 

The connection to the video distribution system may be a cable comiector to a cable 
television dehvery system, asshownmFigure6b. Thecableconnectorincludesadatastripper 



wo 01/67302 



PCT/USOl/04946 



22 

circuit 617, which accq)ts video input from eitherasettop converter, TV or VCR 601, or an 
optional tuner bIock613 thatreceives the CATV signal through flie cable comiector212'. The 
data stripper circuit 617 strips data out of the video, and outputs a digital bit stream to the 
digital logic portion 609 of the library unit 262 . The data is embedded in the video signal either 
in the vertical blanking interval or the active video portion in an encrypted and compressed 
format. The data stiq}per circuit 617 can beplaced inside the set top converter box 601, TV, 
or in the library unit The data stripper circuit 6 1 7 outputs the digital bit stream to be used by 
the library digital logic 609. 

The video connector 212 may also contain a channel tuner module 613 that can tune 
to the video channel and provide access to the video that contains the data to be stripped. 
Using the optional tuner mo d\ile 6 1 3 , a set top converter, VCR, or TV tuner is not needed in 
the home subsystem Theoptionaltunermodule613wouldinsteadreceivetheCATV signal 
direcdy through the cable connector 212. Additional coimector options, which allow for the 
receipt of text files using alternative delivery methods, are presented in Section VII. This 
ubiqmtous access is provided using the modular connector 700 as depicted in Figure 
6b. 

b. Library 

An embodiment of the library 262 for a two unit home-subsystem is shown in both 
Figure 6b and Figure 8. The embodiment shown includes the following optional parts: the 
video connector 212, phone connector 270, RF transceiver 604, and battery pack 624 in 
addition to aremovalportable memory 600', microprocessor 628, instruction mraiory unit 
632, digital logic 636, and power unit 640, 

The library 262 contains a digital logic section 609 (not shown in Figure 8) which 
includes the microprocessor 628, the digital logic 636 and the instruction mraiory unit 632. 
The microprocessor 62 8 may be a secure microprocessor such as the Mot S C2 1 device sold 
byMotoroIa. Thedigjtallogicsection609wiUrecdvethesCTaldigLtaIbitstcea^ 
strippercircuit617andprocesstfaedata. Error correctionwill also bepaformed by the digital 
logic section 609 and the data will be checked for proper address. If fte address of tibte data 
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is correct and the library 262 is authorized to receive the data, the data will be transferred to 
the memory storage unit 600, 600'. Authorization to receive thedataisprovidedbythecable 
headend or anotho* distribution point. An authorization code may be sent in the serial digital 
bit stream. The digital logic section 609 will send appropriate text and graphical data to the 

5 memory storage unit 600, 600'. It transfers this data in a compressed and encrypted format 
and the data remains stored in a compressed and encrypted format, 
i. . Memory Storage Unit 
The memory storage imitofthe library nmybearemovdjleportablememoryimit 600' 
(as shown in Figures 6a, 6b and 8). A variety of options are available for memory storage: 

10 a hard disk drive, such as an 80 megabyte, a 200 megabyte, a hard disk with removable 
platters, and CD ROM. Referring to Figure 6b, ahard disk drive unit 600', which contains 
removable platters, may also be used. This would provide virtually unlimit«i library storage 
capacity. Data will be stored in the memory storage unit in a conq>ressed and encrypted 
format. As is also shown in Figure 6b, the datamay also contain akey orunique ID number 

1 5 that matches the ID or key of the viewer 266 . This matching of a unique key or ID number 
prevents unauthorized transfer of text data from the memory storage unit to an unauthorized 
viewer. Small memory devices such as smart cards, electronic msmory cards or PCM CIA 
cards personal computer memory card industry association) may also be used to store the 
data. 

20 ii. Power Circuitry 

As shown infigures 6b and 8, the library 262 will acceptpower from AC wall power 
610, DC power 640, or optional battery power 624, The power circuitry 610, 640 may 
provide all the vo Itage necessary from either the battery 624 or AC unit fijr the various circuitry 
in the library. Thepower circuitry 610, 640may also providepowerto die viewer through a 

25 single data cable whmcoimected to the viewer. Thepowerciicuitry 610, 640 willrecharge 
fliebatteiyusingACpowerwheninop^ratLon. Withtheoptionalbatteryunit624 installed, 
tiie library 262 becomes aportable unit and can still provide power to the viewer 266. In 
ordCTto extradbatt^ life, powerconservalionmeasuresmaybeutQized, such as shutting 
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down flie memoTy systan when not in use. When the viewer unit 266 is being utilized and the 
library circuitry is not being utilized, virtually all power may be shut down to the 
library 262. 

iii. Connection to the PubUc Telephone System 

5 hi an embodimrat, the connection to the telephone system may be provided by a 

connector device 61 1, which consists of amodem. Various available modems may be used 
. to paform this function. As shown inFigure 6b, cellular phone.orPCNphone cpnnections 
61 1 ' may also be provided. Whm the home system258 is first initialized, themodemmay be 
used to transfer the name and credit card information of the consumer to the billing and 

10 collection system 278. Thetelephoneconnection270maybeutilizedeachtimeanelectronic 
book is purchased by a consumer to complete and record the transaction. The telephone 
connection 270 may also be used to receive the text data from the operations center 2S0, by- 
passing the video distribution system 208 . The phone connection 270 may be a separate unit 
as shown inFigure 6b. However, altematemeansexisttocoimecttfaehome system 258 to the 

15 billing and collection system 278 or the operation center 250. The modular coimector 701 
(shown in Figures 6b and 8) provides access to each communicationnetwoik to provide apath 
from thehome system 258 to the billing and collection system 278 or the operations center 
250. These alternatives are presented in detail in Section Vn. 

iv. Library Processing 

20 Figure 9 shows for one embodiment, an example of processing performed by the 

digital logic section 609 of the library 262 on the data stream 65 1 received from the video 
connector 2 1 2 or stripper circuit 6 1 7. In step S650, digital logic section 609 checks the data 
stream 65 1 for error correction . If an error is detected, in step S654 digital logic section 609 
de-interleaves the data and in step S658 runs aFEC (Forward Error Correcting) algoriflmi. 

25 In steps S650, S654 and S658, the digital logic section 609 performs the error correction 
needed on the data stream. If no error correction is necessary the digital logic section 609 
proceeds to step S662 and checks data packets individually for packet address. 
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If the address is a unique address, &e process moves to step S 666 and the digital logic 
section 609 checks whdher the address of the packet matches the library box JD number. Hie 
library box ID number is aunique number associated with the library 262. The library box ID 
is used to ensure security of the data. The process then moves to step S670 and the digital 
logic section 609 d^ennines A^iiether an electronic file has already been opaied into which the 
datapacket can be saved. If no data file has been opened, the digital logic section 609 opens 
a new data file for that packet. If an electronic file has been opened, die process moves to step 
S678 and the digital logic section 609 saves the packet in the electronic file on disk. The 
process moves to step 682 and the digital logic section 609 checks to see if this is the last 
packet for aparticular book for aparticular textual datablockbeing received. If it is the last 
packet of information, the process moves to step 686 and the digital logic section 609 closes 
the electronic file and iqxiates the directoiy of available electronic files. Following either step 
S682 or S686, the process returns to receive another data packet firom the data stream 
received firom the data stripper block. 

If the packet address is checked and the address is determined to be a broadcast 
address, the process moves to step S690 and the digital logic section 609 determines the type 
of message that is being sent. The message may be an index ofbook titles, menu (and menu 
gc^hics) infbnnation, announcements, special offerings, discounts, promotions, andpreviews, 
for example. Theprocess thenmoves to step S694 and the digital logic section 609 stores the 
message in an appropriate electronic message file. Theprocess then returns to step S6S0 to 
receive another data packet and perform another error check. 

Using the process of Figure 9, the library 262 is able to receive, store and update 
directories related to the textual data and graphical data(thatcanbeusedto depictpictuxes 
inagivenbook or to generate menus). Variations of theprocesses arepossibledepending on 
the format of the data and operating system of the library 262. 

Figure 10showsanexaiiq>leoftfaeprocessingofinfQnnaficmrequestsfrQm&^ 
266 at the Kbraiy 262. Information requests fiiom the viewer266 arerecrivedeitherthrough 
the cable connecting the viewer 266 to the Ubrary 262 or through wireless transmissions such 
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as RF. It is possible in some embodiments for subscribers' requests to come from a set top 
converter box 602 (see Section V). 

InfonnationrBquests received from the viewer 266 generally fall into three categories: 
(1) directory data ofbooks stored in the library 262, (2) index of all available books on the 

5 system, and (3) requests for a specific book (step S700). In step S704, the digital logic section 
609 answ^ a request from ttie viewer 266 for a directory of data showing Qie books stored 
at the viewCT 266. The directory of data is sent to the viewer 266 so that it may be displayed 
to the subscriber. In step S708, the digital logic section 609 handles requests from the view^ 
266 for an index of all available books on the system. The library 262 will obtain an index of 

10 all the available books on the system and transmit that index, in step S712, with menu 
information to the viewer 266. In step S716, the digital logic section 609 replies to arequest 
from the viewer 266 for a specific book. In step S720, the digital logic section 609 opens an 
electronic file for the specific book requested by the viewer 266 and transmits the record or 
transmits the information on a packet-by-packet basis to the viewer 266. This process of 

1 5 transmitting the specific book, record, or packets to the viewer 266 continues until the last 
record or packet has been sent in step S724. 

In addition to the processes shown on Fi gure 1 0 in handling a request for a sp ecific 
book, the libraiy 262 also orders and receives specific books from the op^ations center 250 
using the process as described in step S7 1 6. Following a request for a specific book which 

20 isnotstoi^attfaehl>iaiy262,flielibraiy262wiUproceedtodetemune&^ 

&e book will be on the video distribution system 208 or an alternative delivery system and 
ensure reception and storage of that book (process not shown). In performing this process the 
library 262 will transmit to the viewer information on when it will obtain tiie text data for the 
book so that the subscriber may Adew the book. In addition to timing information, price and 

25 other ordering information may also be passed by the library 262 to ttie subscriber, 
c. The Viewer 

Figurell isablockdiagramofaviewer266showingitsintenialcQmponents. The 
viewCT 266 ofFigure 11 is similar to the viewer 266 depicted inFigure 6b. Theviewer266 
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is designed to physically resemble aboundbook. The viewer266 is made \sp of fiveprimaiy 
conipoiients and seven optional components: (1) LCD display 602, (2) digital circuitzy (not 
shown), (3) video graphics controller 607', (4) controls 740, (5) book memory 728, (6) 
optional power supply circuitry 736, (7) optional battoy 603 ', (8) optional RF transceiver 604, 
S and (9) optional cellular or mobile connector (such as 61 T) (10) optional keyboards 267 and 
268, and (1 1) an optional speakei^a:ophone 608', (12) optional alternative communication 
interface devices. 

(1) AMg^resolutionLCbscreen6Q2,ofVGAquaHty,maybei^ 

266 to display text and graphic images. The screen may be the size of one page of an 
10 electronic book. A two page screen or two screens may also be used with the 
viewer 266. 

(2) IMgitalcin:ui1iy tbatincludes asecuienuc^ioces 

732, and digital logic. Data is transferred to the viewer 266 in compressed and encrypted 
format The secure microprocessor 621 comparesthelDnumberofthe viewer266 withthe 

1 5 incoming data stream and only stores the text data if the ID number of tiie viewer 266 matches 
that within the incoming data stream. The viewer266 maybe configured to not outputtext data 
or other data and that ttie data is decompressed and decrypted only at the moment of viewing 
andonfyforthecurrentpagebeingviewed. These measures provide additional security against 
unauthorized access to data. 

20 (3) A video graphics controller 607 ' that is capable of assisting and displaying 

VGA quality text and graphic images is included in the viewer 266. The graphics controller 
607' is controlled by the digital circuitry described above. Textmay be displayed inmultiple 
font sizes. 

(4) The viewer 266 ofFigure 11 has touch panel controls 740. These unique and 
25 novel controls 740 allow tiie consumer to select stored electronic books and electronic books 
fiom catalogues, move a cursor, and tumpages in an electronic book. Typically, the controls 
740 include forward and reverse page buttons 742, 741, aball743 for cursor movemeaat, one 
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or more selection buttons 745, a cuirent book button 747 and abookmark button 749 (see 
Figure 14a). 

The controls 740 should be easy to use and conveniently located. Referring to Figure 
14a, the controls for the viewer 266 may be located below the screen 602 at the bottom 
5 portion ofthe viewer 266. Thenextpagetumbutton742isthemostusedbutton740andis 
located towards fhe right edge ofthe page. The subscribe is likely to use right hand thumb 
movements to work the controls particularfy 742. Therefore, the 

buttons may be arranged in such a manner fhst the buttons are easily controlled by a 
subscriber'srightthumb. Generally, ttiis canbe accommodated either on die lower portion of 

10 theviewer266(asshown)oralongtherighthandmargin ofthe viewer 266 (notshown). The 
current book button 747 and bookmark button 749 are usually the least used of the controls 
740. Therefore, in the example shown, those buttons 747, 749 are located on the inside 
portion towards the binder ofthe viewer 266Locating the ball 743 or other cursor 
movement device (such as four pointer arrows not shown) in the bottom cento" ofthe viewer 

IS 266isbotheasierforthesubscribertouseandeasierinmanu£aicturingtheviewer266. The 
selection buttons for the cursor 745 may be located below the middle diameter ofthe cursor 
ball 743 on the right and left sides ofthe ball as shown. If pointer arrows are used for cursor 
movement, aselection button 745 maybe locatedinthecenterof the four arrow buttons (not 
shown). Again, themostused controls 740 should be located whereasubscriber'srigihthand 

20 thumb would normally rest. 

(S) Bookmemory 728 for at least one electronic book ormore of text is included 
intheviewer266, 'niemembry728 storestextandany graphics, which represent pictures in 
abook. The memory 728 can also store menu graphics data. Two different memory 728 
devices may be used in the viewer 266, one forthe instructions fortiie microprocessor 621 in 

25 thedigitalcircuitryandasecondtypeofmemoryniaybeusedforthe bookmemory 728 (and 
graphics). Various memory devices available on the market may be used such as, ROM, 
RAM or a srnall hard disk Since an electronic book requires approximately 0.6 megabytes 
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of Storage, R fnnall h^rd di&k providing ^roximatdy 60 MBytes of stoiagepiovides memory 
to Store ^proximately 100 electronic books. 

Text for electronic books may be displayed in various font sizes. To accommodate 
various fonts for display, a variety of fonts are stored in instruction 732 orbookmemory 728. 
5 Thus larger or smaller fonts may be recalled from memory 62 1 , 728 to create displays desired 
by the subscriber. 

(6) .PowCTSiq>plycircuitiy736intheviewer266wiflacceptpowerfo 

Ac power source or ftom an optional battery 603 or the library 2i52. The power supply 
circuitry 736 provides thenecessary voltages to accommodate the various systems within the 
10 viewer 266. 

(7) An optional battery 603 ' is provided in one embodiment Thebattery 603 ' is 
automatically recharged when AC power is available. 

(8) An optional RF transceiver 604 whichprovided two-way data linkbetween 
the viewer 266 and other components of the home subsystem can also be included in the 

15 viewer 266. 

(9) Also, the viewer 266 may include a cellular transceiver (not shown) fiMrmobile 
communications. 

(10) The optional wired (attached) keyboard 267 and wireless (e.g.,RF) keyboard 
268 (see Figure 6a) may be used witti the viewer 266 to provide communications between the 

20 subscriber and the viewer 266. 

(11) Theoptionalspeakerandmicrophone608'allowtheviewer266toprovide 
audio signals to the subscriber, and allow the subscriber to provide an audio input The 
speaker and microphone 608* may be used in conjunction with the cellulartransceiver 608 or 
other telecommunications equipment to provide for reception and transmissicnof telephony and 

25 data. 

(12) The optional alternative communicationinter&ce devices allowtiie viewer 266 
to make use of a variety of conununication paths. 
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Theviewer266ofTigurcllhaspaitsavailableforpro^ alihraiy 
744, electronic caidmOTiory 748, CDROMumts752, and apoitable memory unit 756 (such 
as that shown in Figure 6b as 600'). Various electronic memoiy cards such as PCMCIA can 
be used with the viewer 266 to supply and store electronic books. 

Security, lowpow^consumptionandexceUaDtdisplay tec^ology atedesiiedf^^ 
oftfae viewer 266 design. Theviewer266shouldbelightweig^andportable. Theviewer266 
contains a software operating system that allows electronic books to be stored, read and 
erased and includes the capability to order electronic books andretain them iii memoiy 728 
for a predefined period of time determined by the system operator. The software can be 
configured to allow the electronic book to be read during aperiod of time (i.e., two weeks) and 
then automaticaUy erased, read once and erased, or held inmemoiypermanentty. Each viewer 
266 may have aunique key 605. All of the data storage maybe encrypted with the key 605 
for an individual viewer 266 to prevent more than one viewer 266 accessing the text file or 
electronic book file. 

Figure 12 is a flow diagram of some of fheprocesses executedby the microprocessor 
62 1 in the viewer 266. The viewer 266 may receive inputs from flie subscriber through touch 
panel controls 740. Instq) S800, the subscriber's infoimationrequests are then processedby 
the microprocessor 621. 

In step S804, if the subscriber requests a menu of available electronic books, the 
nucroprocessor621 will select an electronic book menu. Instq>S808,tfaemicroprocessor 
62 1 will open the electronic files that list &e electronic books which are available (related to 
the category of topic of the meau) and display the menu with the names of the available 
electronic books. 

If the subscriber selects a particular book to read, then in step S812, the 
microprocessor 62 1 will process &e selection and determine the electronic file that contains 
the specific electronic book. Instep S816, the microprocessor 621 will open the file for that 
spedficelectronic book andnormally access the firstpage. (If apointerhas already been set 
in that books electronic file, the process may default to ttiat page.) In step S820, the 
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microprocessor 621 will then determine which page needs to be displayed. That is, the 
microprocessor 621 will detemune whether a next page, previous page or abookmadcedpage 
needs to be displayed If the pointer for the electronic file is not in the correct location then in 
step S828, the microprocessor 621 will move the pointer and obtain the previous page of data 

5 fix)mthestoredfile. Othawise,instq) S824, the microprocessor 621 will nomMlly obtain tiie 
nextpageoftextfromthestored electronic file. Instep S832, the microprocessor 621 will 
decrypt and decompress the text data and send the data to the video display. The video 
display will generally have a video display memory associated with it In step S832, the 
microprocessor 621 willsendthedatadirectlytothatvideodisplaymemoiy. The circuitry for 

10 the display then completes the process of displaying the page of text. 

If the subscriber, through the controls 740, requests (fiiom step S800) that the power 
be tumedof^thm instep S836,themicroprocessor621 initiates power off. InstepS840,the 
microprocessor 621 saves the pointer in memory to the page number in the book that the 
viewer 266 is currenfly reading. Instep S844, the microprocessor 621 closes all theelectronic 

1 5 files and signals thepowea* drcuitiy to shut down the power to the various circuits in the viewer 
266. With these examples ofbasic processes the viewer 266 is able to display book selections 
and display text fi:om those electronic books, 
d. Menu System 

Referring generally to Figure 1 3, the electronic book system 200 may have amenu 
20 system 85 1 for selecting features and books firom the electronic book system 200. The 
operating software and memory required for the menu system 851 may be located at the 
viewer266(e.g.,theinstmctionmemoiy732 and/or bookmemory 728). However, it can also 
be located at the library 262 (e.g., the instruction memory 632) or the library 262 and the 
viewer 266 can share the software and memory needed to operate the menu system 851. 
25 Since the menus are usually displayed on the viewer, and since the viewer 266 maybe capable 
of operating in the absence of the library 262, the basic software and memory to create the 
menus is more conveniently located at the viewer 266. 
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Thememi system 851 aUowssequencizigSelweenmemisandpmvi 
for graphical displays such as on the LCD display 602 of the viewer 266. In an electronic 
book system that uses a set top converter these menus may also be displayed on a television 
screen. In an electronic book system that uses a computer, these menus may also be displayed 

S onthecomput^monitor. Inanembodiment,themenusprovidejustbasictextinfoimation 
from which the subscriber makes choices. In other embodiments, the menus provide visual 
displays witti grq)bics and icons to assist the subscriber and allow for subscriber interaction and 
real-time ordering of electronic books or other content available to the subscriber. 

Figure 13 dq)icts the menu system 851 with sequencing. The primary menus in the 

10 menusystemSSl areanintn)ductorymenu850,amainmenu854andvarioussubmenus858. 
In the embodiment shown, there are three levels of submenus 858. In obtain instances one or 
two submenus 858 is suf&cient to easily direct the subscriber to the selection or information 
requested. However, th^ are features in which three ormore submenus 858 make die user 
inter&ce more fiiendly for the subscriber. Each level of submenus 858 may consist of multiple 

15 possiblemenus for display. Theparticularmenu displayed depends on the selection by the 
subscriber on the previous shown menu. An example of this tree sequence of one to many 
menus are the help submenus 887, 888. Depending upon the specific help requested, a 
different level two help menu is displayed to the subscriber. 

An example of an introductory menu 850 is shown on Figure 14a. Generally the 

20 introductory menu 850 introduces the viewer 266 to the system and provides initial guidance, 
announcements andinstruction. Tbeintroductorymaxa850isfollowedbyamainmenu854, 
anexampleofwhidiisshowninFigurel4b. Themainmenuprovidestheviewar266withttie 
basic selection or features available in the system. Figure 14b is an example of amainmenu 
854 offering many additional features and submenus 858 to the subscriber. For example, 

25 Figure 1 4b shows that the view^ 266 is able to choose by a point and click method, many 
options including: (1) fiee previews, (2) books you can order, (3) books in your library, (4) 
your current book, (5) help, (6) on-line services and (6) other system features. Following a 
selection on the main mem 854, a corresponding submenu 858 is shown. 
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Figure 13 shows fourteen available primaiy or first level submenus. They are (1) 
account set up 862, (2) fiee previews 866, (3) book suggestion entries 855, (4) books in your 
library 872, (5) books you can order 878, (6) your current book 884, (7) help 887, (8) 
available features 890, (9) messages 893, (10) account infomiation 896, (11) outgoing 

S message submenu 898, (12) show links submmu 970, (13) create links submenu 980, and (14) 
show interactive files submenu 990. Figure 14c is an example of a first level submenu for 
books in your library 872. This **BookIn Your Library" example subnaenu 872 shows six 
available books by title and author and provides the subscriber with the ability to check a 
different shelf of books 874 or return to the main menu 854. Figures 14d and 14e show 

10 ©cample submenus 858 for books that may be ordered using the "Books You Can Order" 
submenu 878. 

Figure 1 4f is an example of a confirmation menu which confirms a subscribers order. 
In this particular example, tiiie subscriber is required to ^ter a PIN number to complete the 
subscriber's order. Any alpha-numeric or similar password may be used to ensure the 
15 subscrib^is an authorized subscriber. In one embodiment, the subscribe confirms an order 
with a PIN or password and then receives a final confirmation screen. The final confirmation 
screen is primarily text and may state: 

Year book order is now being processed using CABLE, 

Your book will be delivered overnight and your VISA account will be charged 
20 S2,9S . 

Your book will be available for reading at 6;00AM EST tomorrow. Make sure 

that: 

1. your Library Unit and Cable Connection Unit are plugged in with aerials up 
tonight; and 

25 2. you tune your cable converter to THE BOOK Channel The TV set does 

not have to remain on. 

or similar language. 
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Examples of the "Account Set Up Menu" 862 and further submenus 858 related to 
account set vp (which provide instructions and account input 864) are shown in Figures 1 4g 
andFigure 14h. These submenus 858 aUow initialization ofan account at the operatioiiscato 
250 and orders to be charged to credit cards. The submenus 858 include the abihty to enter 

5 datarelatedtoyomdesiredPINnumberorpassword,creditcards,phoneniimbers,etc. In 
oneembodiment,theaccountsetupbep^onnedusingthetelq)hone system. Aconfirmation 
menu verifies that the account has beenproperly set up with the desired PIN orpassword and 
credit card However, additional set-up methods are presented in Section VII. 

Free previews for books 866 are also provided by submenus (868, 870). Examples 

10 ofthefreepreview menus are shown mFigure 14iandFigure 14j. Figure 14ishowsamenu 
depicting various books for which previews are available for viewing. Following a book 
selection, a screen submenu showing an excerpt of the selected book cover's description is 
provided along with an excerpt from a critic's review of the selected book. In one 
embodiment, this preview screen for aparticular book also allows the subscriber to select a 

15 submoiuwhichprovidesinfoimationabouttheauthor. Thebookpreviewsubmenumayalso 
include a still video picture or gr^hics portraying abook cover or a scene from the book. An 
example of sudi a sdll video picture or graphics is shown in figure 14j which depicts apreview 
screen 870 about the author. The author's preview screen 870 shows apicture of the author, 
provides a short biogr^hy, and may allow the subscriber to order the author's books. The 

20 price for ordering the authors various books may also be shown on the menu. 

In addition to ftee previews, in other embodiments, the electronic book system 200 
provides the subscribe: with abook suggestion feature (see 855). This is accomplished using 
the menu system 85 1 and the processor with associated memory located at the viewer 266, 
library 262 or at the distribution point (1020 or 250). When necessary, information for the 

25 booksugges1ionfeatureissentin1faetextdataofthesignaltodiehomesyst^258. Withthis 
feature, books or au&ors are suggested to a subscriber based upon historical data of fiie 
subsctiber'sprevious oide^, draiogr^faics or mood of the subscriber, o&erindicators, and/or 
by text word searches. 
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Iq one book suggestion embodiment, tert word searches of pieview infonnadon (such 
as book cover descriptions, critics reviews and bio graphics about the author) and/or text of 
books or other titles are performed by the library 262 using databases stored in the Hbrary 
memory 600. Personalizedbook or author suggestions aremadeto the subscriber by obtaining 

S infomciationfromthesubsoiberindicativeofgeneralsubscriberinterest^ Subscribe oitries 
may be solicited fix>m the subscriber using the book suggestion entry submenu 8SS. The 
system uses these subscriber entries either directly or indirectly to search for books or authors 
to suggest to the subscriber. 

Generally, the electronic book suggestion methods may be categorized into two 

10 categories, either responsive methods (whichrespond to aseriesofsubscribermenu entries), 
or intelligentmethods (which analyze data to suggest abook). Using aresponsive or inteUigent 
metiiod, the system 200 detennines a list of suggested titles or authors and creates a second 
or third level submenu 856, 857 to suggest the titles for subscriber selection. 

Responsive metiiods of suggesting titles include, for example, the use of mood 

15 questions, searching for authors, andkeywordseaiching. Usingtheinstructionmemory732 
and menu generation hardware (e.g., 607) of the viewer 266, a series of mood questions can 
be presented on menus to determine a subscribers interest at a particular time. For this 
methodology, the operations center's 250 processor 404 and instmctionmemoiy 416 assign 
each title mood indicators (and sub-indicators) &om a group such as light, serious, violent, 

20 short, long, dull, exciting, con^lex, easy-read, young theme, old theme, advoiture, romance, 
drama, fiction, sdaice-fiction, etc. TheseindicatorsaFesenttothehomesystem258with1he 
text data and are stored in library memory 600. Based upon the subscriber entries, the 
processor associates a set of indicators with the subscriber's request and a set ofbooks with 
matching indicators are located for suggesting to the subscriber. 

25 Responsive searches for authors or keywords (a search word provided by the 

subscriber) are generally perfomied by the library processor 628 and instruction memory 632 
on data stored in the library memory 600. For example, akej^ord given bythe subscriber 
may be searched for a match in library memory 600 storing tiie book reviews, critics and 
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previews databases. Thus, if a subscriber provided an entry of the word "submarine*' on an 

^propriate submenu, the title 'Hunt For Red October" maybe located by the microprocessor 

628 usiug instruction from a routine in instruction manory 632. 

bitelligent mdhods of suggesting programs include analyzingpersonal profile data on 
S tiie subscriber and/or historical data about the subscriber such as past books ordered by the 

subscriber (or buy data). This me&od may be performed at tiie distributionpoint or operations 

center 25 0 by the on-site processor 404 using subscriber databases stored in memory 428. 

The home system 25 8 receives flie text data including program suggestion information &om the 

distributionpoint or operations center 250 and generates the program suggestion submenus 
10 855, 856, 857usingthesametextdatareceiving212 andviewermenugenerationhardware 

(e.g., 607, 621) described above. Software routines and algorithms stored in instruction 

memories (e.g. 632, 732) are used to analyze historical data and book ordered data to 

determine a line of books to suggest to the subscriber. 

The algorithms for this powerfiil feature of suggesting books or authors to subscribers 
15 is disclosed in great detail in U.S. Patent Number 5,798,785, entitled TERMINAL FOR 

SUGGESTING PROGRAMS OFFERED ON A TELEVISION PROGRAM DELIVERY SYSTEM, filed 

December 2, 1993, which is incorporated herein by reference. 

Referring to Figure 1 3, submenus 85 8 are shown on the **Books Jn Your library" 

submenu 872 and maybe broken into shelf numbers with submoius for each shelf 874, 876. 
20 The submenus 858 for the ''Books You Can Order" submenu 878 is similarly broken out into 

submenus by shelves 880, 882. These shelves may each be a category or genre ofbooks. 

Books may be gmiq)ed into categories such as best sellers, novels, fiction, romance, etc. See 

Figure 14d. 

Referring to Figure 13, the submenu 858 for **Your Cuirmt Book" 884 allows a 
25 subsciibertoselectacutrCTtbook884andthendetemunewhatpagetoview. Thisselection 
is confirmed with a level two submenu 885 . The help submenu 887 provides the subscriber 
witiiadditionalhe^ screens 888. Thesubmenus 858 for available features 890may be brokra 
out into a sequence of separate submenus for each feature 891, 892. 
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Referring to Figure 13, messages can also be sent witti the electronicbook selection 
and delivCTy system 200. Alevel onemessage screen provides the subscriberwifli the ability 
to select from variousmessages the subscriber has pending 893. Each message is then shown 
on a separate submenu screen 894, 895. The message may contain text and graphics. 

S Refetringto Figure 13, account informationis shown on alevel one submenu 896 and 

then follow-on submenus 8S8 show the recent orders and your account balance 897. There 
is also a level one submenu for outgoing messages 898 which has a foUow-on submenu used 
as an input screen 899. 

Inadditionto the specific features and submenus describedinFigure 13 andFigure 14a 

10 throughFigure 14j,manyottiervariations and features are possible. When abook is finally 
selected for viewing the title page 886 will appear on the screen followed by a page 
of text. 

m. BilUng And Collection System 

Inoneembodim^lhebilling and collection system 278 (showninFigures2and3) 
1 5 utilizes the latest technology in electronic transaction and telephone switching to tack orders, 
authorize deliveries, bill consumers, and credit publishers automatically. The telephone calls 
initiated by the phone comiector270 are received by the billing and collection system 278 
wUchresponds immediate^ without human interve^^ 

consumers credit card account Data is compiled periodically and publish^ 282 are credited 
20 forsalesoftheirbooksorothertext The billing and collectionsystem278 may also connect 

with subscribe throu^ two-way cable a>nnections,ceUular,oroft^ 

These additional methods are detailed in Section Vn. 

The billing and collection system 27 8 communicates with the op erations carter to track 

changes in available books and to provide statistical data to the operations center 2S0. 
25 IV. Public Library, School, and Bookstore System 

The electronicbooksystemcanbemodifiedtobeusedatpublic libraries, schools, 

bookstores,newsstands,orstand-alone kiosks. Figure IS shows onepossibleanangCTient 

ofcompon^fortfaedistributionlocalion. TheiDainumtis1hefilesavear900. Thefileservo: 
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900 is a large electronic memoiy unit that can store thousands of books, newspapers, or 
periodicals. Variouselectronicstoragemeans maybe usedin&efilesenrers, suchashard 
disks, read-write CD ROMs and read-only CD ROMs. 

The system comprises five components; the fiile server 900, a converter or video 

5 connector 904 or connector capable of interfacing to one of the alternative dehvery systems 
presented in Section vn,' a controller 908, a viewer 912, and a catalog printer 916. The 
software for controlling the system is primmly located i^ The converter or 

videoconnector904issinulartothosedescribedab6ve. In this configuration the controller 
unit 908 monitors the data being transferred to the file server 900 by the converter 904. The 

10 controller 908 may beprovided with a viewing screen and several control buttons. Whenit 
is necessarytohavealarger screen to perfomimore sophisticated controlling of the systema 
viewer 266 may be connected to the controller 908 and the viewer screen and controls 740 
may be used. 

For security reasons, die controller 908 is only able to download books to public 
15 viewers 912 which are authorizedtoreceivebooks from tiieparticular file server 900. Also 
for security reasons it is not desirable that the public viewer 912 have access to more ttian one 
file server 900. In this way, security can be maintained over the text data for books. The 
public view^ 912 maybe limited to receiving one or two books at a time from the controller 
908 . When the user of the public viewer 912 needs anew or additionalbook, the user retums 
20 the viewer 912 to theschool orpublic library where die userreceives a new book from the 
controller 908. 

In order to track the books that are available on the file server 900, flie titles of the 
available books may be printed on acatalogprinter 916. The catalog printer 91 6 is connected 
to the library controller 908 and the titles of the books are downloaded to flie catalog printer 
25 916. For security reasons, the coded text for any of the electronic books may not be 
authorized forprinting using the controller 908 andcatalogprinter916. In orderto maintain 
security over the data, none of the electronic book data may be allowed to be downloaded to 



wo 01/67302 



PCT/USOl/04946 



39 

diepxinter 916. Once acompletepzintout of available book titles, magazines, orotfaer textual 
material is complete, ahard copy of the catalog 920 can be maintained at the file server 900. 

The system shown may also be used at bookstores. The bookstores can rent the 
public viewer 9 1 2 to customers with the text for one or two books loaded onto the public 

5 viewer912. ThepubUcview®912maybeprovidedwi&anautomatictnneoutsequence. The 
timeout sequence would erase the textual data for the books after a certain period of time, for 
example, two wedcs. It is expected that after ap^odoftime(p^h29swittm two weete^ 
renterwouldretumfhepubUcviewer912 to liie bookstore ahdreceiveadditio for 
viewing. Using this arrangement, it is also possible for the bookstore to permanently) sell a 

10 viewer 9 12 to aregular customer. The customer then returns to the bookstore from time to 
time to receive textual data for a book which the customer can then store pemianently on the 
customer's own viewa:912. Various other configurations arepossible for bookstores, schools 
and public libraries using the file server 900 and public viewer 912 described. 
V. Use Of A Set Top Converter 

1 S Existing set top converter boxes such as those made by Scientific Atlanta or General 

Instruments are presently uneqmpped to handle the book selection system of the present 
invention. Although set top converters may be built which include the library functions, 
hardware modifications are necessary in order to use the book selection system with existing 
set top converter technology. 

20 Figures 16aand 16b are examples ofhardware modifications onq)grades. Aport is 

used to attach hardware iq}grades described below to a set top terminal. Two upgrades are 
possibletosettopconv^ers601 toassistinrecdvingandselectmgelectronicbooks, amenu 
generation card upgrade (Figure 1 6a) and an information download unit (Figure 1 6b). Each 
of these upgrades maybe coimected to the set top terminal unit through an upgrade port. A 

25 fourwire cable, ribbon cable, FireWire (IEEE 1394B) interface connector, USB connector, 
or the like may be used to connect the upgrade to the set top converts 601. 

A card addition 9S0 to a set top converter 601 is depicted in Figure 16a. The card 
950 shownprovides &e additiozvdfisictionality I 
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existing set top converts 60 1 technology. The card 950 may be configured to slip inside the 
frame of a set top terminal and become part of die set top terminal, an advanced set top 
tenninal. The primary functions the card 950 adds to the set top converter 601 are the 
interpreting of data signals, generating of menus, sequencing of menus, and, ultimately, the 

5 abilityofthesubscribertoselectabookusingeitherthetelevisionoraviewer266. Thecard 
950 also provides a method for a remote location, such as the cable headend, to receive 
information on books ordered The books ordered iiiformadon and control commands may 
be passed fiom ttie cable headend to the card 950 using telephone lines or alternative ordering 
methods as presented in Section Vn. 

10 The primary components of the card 950 are a PC chip CPU 952, a VGA graphic 

controller 954, a video combiner 956, logic circuitry 95 8, NTSC encoder 960, a receiver 962, 
demodulator (not shown), and a cotmector 61 1', which consists of a dialer. The card 950 
operates by receiving the data text signal fiom the cable headend through the coaxial cable. 
The logic circuitry 958 of the card 950 receives data 964, infirared commands 966, and 

15 synchronization signals (not shown) fiom the set top converter 601. Menuselectionsmadeby 
the viewer 266 on the remote control are received by the set top converter's 60 1 IR equipment 
and passed through to the card 950. The card 950 interprets the IR signal and determines the 
book (or menu) the subscriber has selected. The card 95 0 modifies the IR command to send 
the information to the set top converter 60 1 . The modified IR command contains the channel 

20 infonnationneededbythesettopconvertCT601. Usingthephoneline968anddialer6ir, 
&e card 950 is able to transmit electronic books ordered information to the cable headend 
It is also possible to receive the electronic books over the telephone lines and other 
telecommunications networics, including wireless networks, and by-pass flie video distribution 
system. 

25 These cormnands are passed through the interface linking the set top terminal's 

microprocessor with the microprocessor of die hardware upgrades. In this way, subscriber 
inputs, entered finou^ the set top tenninal keypad or remote control, can be transfened to any 
of the hardware iq)grades for processing and responses generated th^ein can then be scat 
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back to the set top tenninal for display. In one embodiment ttie IR commands 966 are 
transferred from set top terminal 601 to hardware upgrade. 

Hardware upgrades may include amicroprocessor, interactive software, processing 
circuitry, bubble memory, and a long-term memory device. In addition to these basic 
S components, tiiehardware iq)grade may make use of an additional telephone modem or CD- 
ROM device. 

An information download hardware upgrade 1001 showninFigure 16b allowsthe 
subscriber to downloadlarge volumes of information from the operations centdr2S0 or cable 
headend using a set top tenninal 6 10. The hardware upgrade 1001 will enable subscribers to 

10 downloaddata,suchaselectronicbooksandelectroriicniaga2ines,tolocalstorage. Primarily, 
the hardware upgrade 1001 is an additional local storageunit 1003 (e.g., hard disk, floppy, 
optical disk or magnetic cartridge and may include a microprocessor 1 005 , instmction m^ory 
1 007, and arandom access memory 1 009, as shown in Figure 1 6b). A small portable viewer 
may also provided with the upgrade 1001 to mable downloaded text to beread without the 

15 use of a television. 

The downloadable information may be text or graphics supplied by the operations 
center 250 or cable headend. With the upgrade 1001, electronic books may be downloaded 
and read anywhere with the viewer 266. Using tiie upgrade 1001, electronic booksmaybe 
downloaded and stored in compressed form for later decompression. The electronic books 

20 may be decompressed only at the time of viewing. Important text that the public desires 
immediate access may made available tiirough this system. Text such as the President's 
speech, anew law, or arecent abortion decision rendered by the Sipreme Court may be made 
immediately available. 

In one embodiment, electronic book ordering information is stored at each set top 

25 temiinal610untilitispoUedbythecabl6headendusingapollingre^^ An 
exampleofapolliiigrequestmes5agefi)rmatcorisistsofsixfields,iiamely: (1) a leading flag 
at the beginning of tbemessage, (2) an address field, (3) asubscriber region designation, (4) 
a set top tenninal identifier that includes a polling command/response (or P/F) bit, (5) an 
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infonnation field, and (6) a trailing flag at the end of the message. A similarresponse firame 
' foimatforin&imationcoxnmiinicatedbythesettop 
to the polling request may be used. 

Figure 1 7 shows components of a set top terminal 610'. The components include a 

S datareceiver617' andadata transmitter 1011. The data transmitterprovides upstream data 
CQiQmumcatioDScq)abifity between the settoptemiinal Upstream 
data transmissioiis are accompUshed using the polling system described 
transmitter 101 1 . Both receiver 617' and transmitter 101 1 may be built into the set top 
teroimal 610* itself or added through an up grade module. Regardless of the specific hardware 

10 configuratioii, die set top terminal's datatransmissionc^abilitiesmay be accomplished using 
the hardware shown in Figure 17. 

Figure 1 7 shows RF signals, depicted as being received by a datareceiver 617' and 
tuner 613 working in unison. Both of these devices are interfaced with the microprocessor 
1013, whichreceives mputs lOlS, fix)m the subscriber, eitherfhrougha settop terminal's 

IS keypad, aremotecontrolunitorviewer266. Generally, all cable signals intended for reception 
on the subscriber's TV are accessed by the tuner 613 and subsequently processed by the 
processing circuitry 1017. This processing circuitry 1017 typically includes additional 
con^XHients (not shown) for descrambUng, d^odulation, volume control andremodulationon 
a Channel 3 or 4 TV carriCT. 

20 Data targeted to individual set top terminals is received by the data receiver 617' 

according to each set top terminal's specific address or ID. In fiiis way, each addressable set 
top terminal 610' only receives its own data The data receiver 617' may receive set top 
tooninal 610* specific data in the infonnation field of the signal fi:ame described or on a separate 
data carrier located at a convenient firequency in the incoming spectrum. 

25 Therecdveddata]nctudesiiifi>rmationregardiQg electronic books andmenus available 

forselectioa Thesubsaibermayenterasoiesofcommands 101 S using akeypad or remote 
control in order to choose an electronic book or menu. Upon recdpt of sudi commands, the 
set top tenninal' s microi^rocessor 1013 instructs the tun^ to tune to the proper firequency of 
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the channel cairying data and subsequently instructs the processing circuitry 1017 to begin 
descrambling of this data. 

Upon selection of an electronic book, the microprocessor 1013 stores any selection 
infoimadon in local memory (not shown) for later data transmission b ack to the cable headend. 

5 The set top terminal's microprocessor 1013 coordinates all CATV signal reception and also 
interacts with vaiiousupstreamdatatransmissioncompQnen Typically, flie data transmitter 
1011 operates in the return frequency band between S and 30 MHZ. In an alternative 
embodiment, the frequency band of lb to ISMHZmaybeused. Regardless, however, of the 
frequency bandused, the data transmitter 101 1 sends information to the cable headend in the 

10 information field oftfie response frame described. Those skilled in the art will recognize that 
a number of variations and combinations of the above-described set top terminal hardware 
components may be used to accomplish upstream data transmissions. 



VI. Books-On-Demand SystKn 

The electronic book systan 200 described may also be configured in a book-on- 

1 5 demand style. Figure 1 8a shows one example of a configuration for a book-on-demand 
system. Abook on demandsystemrequiresmorepowraM two-way commumcalionsbe^ 
the consumer's home, bookstore, school orpublic library and either the operations center 2S0 
oradistribution site 1020 suchas&ecable headend. In one embodiment, this type of two- 
way communication can be provided by ttie hardware shown in Figure 17 and described 

20 above. Additiorialmethodsrelatedtoalternativecoinmiaucationpalhsaiepresente^ 

vn. 

Referringto Figure 18a, inabook-on-d^nandsyst^thesubscriber selects the book 
tobedowidoadfromanavailablemenuofbooks(seefQrexampleFigures 14dandl4e). The 
data for menus of available books is usually s^ to the subscriber location by the distribution 
25 site 1020. Afterthesubscriber'smmuselection,informationaboutthesi^ 
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request) is then commuziicated to dtheradistiib 1020 (suchasacable headend) or 

the operations center 2S0. Upon receipt of this request, the needed textual and graphical 
information for tfaehookis spooled and sent to the subscriber. In this manner, electronic books 
are only sent whm requested by the subscriber and are sent inmiediately iq>on demand for tiie 

5 electronic book (or text). 

In order to support such a demand system, the text deUvery and distribution must be 
conducted on a strong nodal architecture distribution system, such as, a video-on-demand 
cable or telephone television system, throu^ use of individual telephone calls on 
telephone system or cellular phone system, through the useof (he Internet, or anumber of otiier 

10 data network options. 

The bookHDn-demand system allows for a greata: selection of electronic books to the 
subscriber and limits flie amount of communicatedbook data that is unnecessary or unneeded 
It also provides the electronic book to the subscriber in a much timelier fashion. 

In addition to a stronger distribution system, a book-on-demand system requires a 

1 S distribution point 1020 to have more sophisticated equipment to access and "spool ouf ' the 
textual infbimatioa ThiscanbeaccompHshed using jHeservertechnology 1024 forstoringthe 
booksanddistributiontechnology suchas ATM 1028 or telq)hone-type switching (not shown) 
to distribute tide textual information. The file server 1 024 and distribution technology that can 
be used in configuring such a book-on-demand system is described in U.S. Patent No. 

20 5,262,875 and U.S. Patent 5,218,695, cited above. 

Figure ISashowsanembodimentforabook-on-deniandsystemtfaatutil^ 
technology. In addition to books, ttie embodiment ofFigure ISawill support distribution of 
nearly any digital data. Books or tactual files are received firom pubUshers 282 and other 
sources through local feeds 1032, ATM 1028, orby satellite dish 1036, for example. Thedata 

25 isthenstoredinmemoryl040atthefileserverl024. In one embodiment, the distribution 
point 1020 is a cable headend that receives requests firom subscribers and delivers text to 
subscribers over a two-way communication system (such as a video-on-demand system 
(VOD)1044). 
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The lihracy 262 canbe connected to either abasic piemium-type service cable system 
1048,anear videoH)n-demand type cable system (orpay-per-view^PV)1052)orav^^ 
onrdemand cable system 1044. La connecting with either of these three sj^ems the library 262 
may access the cable directly ormay access the system through a set top terminal 601 601 
5 or 601'". 

Usingthetwo-way video-on-demand system 1044, a subscriber is able to request a 
spedficbooktitleandreceivethattextinmxediatelyM To accomplish this, 

Ifaedistributionpoint 1020 transmits alist of available books through the cable deUvoy system 
to the library 262. The library 262 displays the list of available books on amenu or similar 

10 format As described earlier, the library 262 may use menus which list categories of available 
books to form its request from the distribution point 1020. Afterselectingabookthe library 
262 then sends a request signal on the two-way communication system 1044 back to the 
distiibutionpointlQ20. Thisrequestsignal can behandledin two ways. The Hbraiy262 either 
initiates the request or the distribution point 1 020 polls the various libraries on to the two-way 

IS system 1044. lJlponreceivingtherequestfi)rftebooktitle,thetextassociatedwiththatbook 
title is transmitted to the library 262 using the two-way cable system 1044. 

Figure 1 8b is an expanded view of an operations center 250 that supports aregional 
or national book-on-demand system. In fact, the operations center 250 shown supports 
distribution of nearly any digital data. The operations center 250 supports multiple feeds to 

20 receive digitalinfoimationbyt^e 1060, 1060', ATM 1028, or satellite 1036. The information 
is processed through an input MUX 1 064 and a small file server 1 068 before reaching the 
noast^ jSle serv^ 1 072. Digital data such as books recdved fiom publishers 282 is then stored 
on the master file server 1 072. The digital data may be stored compressed ia a standard 
format such as MPEG2. 

25 A system controller 1 076 provides control over the regional or national book-on- 

demand system. Books may be packaged into groups to provide feeds to various cable 
headends. In addition, schedulingand marketing research are conducted at the operations 
center 250. Ihordertohandlethe scheduling andniarketresearch,bookbiiydataisreceived 
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at the operations center 2S0 through a multiplexer 1082. Book buy information can be 
provided by the operation center 250 to the billing and coUection system 278. 

The opCTations center 2S0 is also equipped to insert messages or advertisements into 
the file server. These messages or advertisements will eventually be received by the 
5 subscribers. 

The master file server 1 072 uses an output multiplexer 1080 and ATM 1 028 as well 
as satellite connections to distribute digital data. In one embodiment, cable headendsreceive 
textdataonbooksfix)mtfaemasterfileserver 1080 through the output multiplexer 1028 and 
an ATM system 1 028. After receiving the digital book data, the cable headends store the 

10 books in alocal fileserver 1024. Figure 18a's distribution pomt 1020 is an example of a cable 
headend which may receive data from the operations center 250 of Figure 1 8b through an 
ATM hookup 1088 or satellite hookup. 
Vn. Altemative Delivery And Ordering Methods 

Electronic books and related data, including electronic book menu data, may be 

1 5 provided to subscribers by use of an on-demand delivery system in which electronic books are 
delivered after an order is received by the delivery system. The delivery system may supply 
the electronic books in real time or near-real time (i.e., near on-demand), or after a delay 
period that allows the delivery system to process, package and transmit the electronic book. 
Altonatively, the delivery system may broadcast oneormore electronic books inacohtinuous 

20 &shion. Irithi5altemative,thesubscribCTindicatesadesiredelectit>nicbookfix>maM 

broadcast electronic books. Iliedeliverysystemmayincludeabillingsystemthatdebitsa 
subscriber's account, or debits acredit card, for example, iq>on delivery of the electronic book. 
The delivery system, or arelated authorization system, may provide alocal authorization code 
that allows the subscriber to decrypt, store and view the desired electronic book. Altemative 

25 delivery systems and methods are disclosed in pending U.S. Application Saial No. 
09/289,956 entitle4ELECTR0MCB00KALTEKNATIVEDE^ 
filed April 13, 1999, andU.S. Application SerialNo. 09/427,938 entitled, VIRTUAL ON- 
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DEMAND ELECTRONIC BOOK, filed October 27, 1999, the disclosure of which are 
incorporated by reference. 

a. Internet Delivery Methods 

Figure 19 is an alternate delivery plan 301 * that provides for electronicbook delivery 
using the Intemet. In Figure 26, the publishers 282 provide the electronic books to be posted 
atthelntemetweb site 279. Thepublishersmayconvertthetextand graphical data todigital 
format, compress the digital data, and upload decompressed digital data to the Intemet web 
site 279. Alternately, the publishers 282 may arrange for an outside conversion activity 283 
to convert the text and gr^hical data to digital format. The conversion activity 283 may flien 
provide the digital data to the Intemet web site 279. For example, a large on-line bookstore 
could gather publications in electronic fomi from a variety of publishers, or could convert hard- 
copy books to electronic form, and post the electronic books on the Intemet such as at the 
Intemet web site 279, 

The electronic books may then be transferred using a public switched telephone 
network (PSTN), for example, or other communications systems, direct to asubscriber 285, 
a library 286 and a bookstore 287 . The Ubrary 286 and the bookstore 287 may also provide 
electronic books to the subscriber 285. 

When electronic books are provided by the Intemet web site 279, the billing and 
collecting functions may be incorporated into the Intemet web site 279. For example, a 
subscriber may pay for an electronic book selection by entering a credit card number into a 
data field of a page of the Intemet web site 279. In this configuration, a separate billing and 
collection system may not be required. Alternatively, the Intemet web site 279 may 
communicate information with the billing and collection system 278. 

Electronicbookdeliveiyoverfiielntemetmay be handled using anumb^ofmethods. 
In a method, the electronic book may be downloaded to the requesting home system 258 
immediately .after the order has beenprocessed. AltCTiatively, flie electronic book may be e- 
mailed to an e-mail address that is entered as part of the ordering process. In anothCT 
embodiment, as part of the ordering transaction process, the subscriber is provided with 
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location and authoiization infonnation that allows the subscriber to retrieve the ordered 
electronic book at tiie subscriber's convenience. For delivery of subscription electronic book 
products, like newspapers, magazines or otiier periodicals, tiie Internet web site 279 can 
deKvertiie latest veasion of Iheproductto the subscriber automatically immediately upon logon 
by the subscriber to the Internet web site 279 . Electronic book data may also be embedded 
into continuous multicast streaming video, audio, or data feeds. 

Delivery methods that allow for dedicated, full time dehvery can b e used to provide 
continuous distribution of electronic book data including requested electronic books fidm 
subscribers, electronic books to be broadcast to aU subscribers, updatedmenu contents, and 
updated advertising. For deUverymethods that allowfornon-dedicated or user established 
connectivity, the operations center 250 may manage the timing and delivery of content by 
delivering the electronic book only ^en requested, orperiodlcally, to ensure delivery for those 
home systems that may not be able to receive the deUvery. Finally, delivery methods that are 
capable of two-way conmiunication may be used to provide a retum path to the operations 
center 250 or billing and collection sj^tem 278 for the purpose of ordering or requesting 
updated electronic book information. Two-way communications paths may be used to allow 
linking fiom an electronic bookhome system 258 or viewer 266 to external nodes &at provide 
world watch live content These delivery methods are described in detail in co-pending U.S. 
PatentAppUcationSerialNo. 09/289,956 entitled ELECTROMCBOOKALTERNAT^ 
DEUVERYMETHODS, filed^ril 13, 1999, fliedisdosureofwhich is hereby incorporated 
by reference. 

Additionally, allfaoug^flie home system 258 ispresented as anindq)end€nt device that 
directly interfaces with the delivery system 200, the home system 25 8 may cormect to the 
delivery system 200 througji a set top terminal, TV, PC, radio, or any other device capable of 
receiving the signal provided by the deUvery system 200. Additionally, the home system 25 8 
may not be an independent apparatus, having some or all of its fimctionahty supported within 
the set top terminal, TV, PC, radio, or any other device capable of receiving the signal 
provided by the delivery syst^. 



wo 01/67302 



PCTAJSOl/04946 



49 

b. Alternative Ordering Methods . 

Any of the delivery methods described in the section above are viable embodiments 
forproviding access from thehome system 258 to the billing and collection system 278 or 
operations center 250. In an embodiment, the modular connector 701 , as showninFigures 

5 6b and8,maybeiisedtoprovidethespecificprotocolfom[iattingandtransmis^ 

to allow the home system 25 8 to use the conmiunication path. In the embodiment where the 
PSTNisusedtopK)videaccess,fliemodularconnector701 includes the phone connector 270 
and the modem 611 as depicted in Figure 6b. Intheembodimentwherethecellularphone 
systemis used to provide access, the modular connector 701 includes the cellular phone or 

10 PCN phone 61 T as depicted in Figure 6b. 

c. Mobile Environments 

A feature providedby delivery methods using a wireless broadcast system, satellite 
broadcast system, wireless personal communication system, or ten?estrial televisionbroadcast 
systemismobilityof1heelectronicbookhomesystem258arviewer266. This mobility allows 

1 5 for ordering and receiving electronic book data anytime or anywhere, from sittmg on abeach 
in Florida to sitting on a bus in New York City. This mobility allows for the delivery of 
electronic book data subscription products such as daily newspapCTS, monthly magazines, or 
books from book-of-the-month clubs. These subscription products may be delivered 
automatically to the ^propriate home syst^ 258 or electronic book viewer 266. Enhanced 

20 end-to-end error correction techniques can be added to the transmission system to ensure 
hig^probabilityofreceq)tforthesemobileenvironments. Additionally, transmissionmethods 
may be implemented that resend packets of electronic book data, changing their delivered 
order on each resend, to improve likelihood of receipt. For lower bandwidth mobile 
envirormients, an electronic book file may be broken up into packets and the p ackets sent a 

25 limitednumber of times. Ifthe electronic book file is not received coiiq)letely,tiie electronic 
book viewer 266 may initiate a request to the operations center 250 to resend only tfiat portion 
of the electronic book file yet to be received. 
Vin. Electronic Book Link Svstem 
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Electronic book links allow the subscriberto use the electronic book viewer 266 to 
traverse pre-defined paths between contmt in their currently viewed electronic book to related 
information contained either elsewhere in the electronic book, elsewhere on the viewer 266, 
or external to the viewer 266, including in the library unit 262 or in a connected Internet web 

5 site. These links provide an organized and methodical method for the subscriber to quickly 
access related topic areas or seek clarification of the currently viewed material. 

An electronic book includes first locations, or components, such as words, phrases, 
sentences, sections oftext, paragraphs, pages, chsqiters, figures, drawings, maps, video clips, 
and audio clips. Links to second and subsequmt locations, or components, contained in the 

1 0 same electronic document or in another related electronic document, file, or database can be 
associated with each of these first components. First components with underlying links canbe 
highlighted and displayed on the viewer display 602 or on the connected television 259 or a 
personal computer261 (seeFigure2). Firstconqxmentsthathaveunderlyinglinks associated 
withthemniiaybeidentifiedbyassigcungtfaemauniqueidentifier. Theuniqueidentifiercanbe 

15 awoniorphrase,andpha-numericvalue,acoordinatepoint,orotheriD3iqueidentifier. Inan 
embodiment, each such first location may be assigned an identifying index value. 

The use of the index value allows the first components to maintain links with other 
components, even if the electronic book is altered. For example, a subscriber may use a cut 
andpaste edit feature to move ablock oftext containing a first component Cutting and pasting 

20 will not afifect the status of the first component and its links to other components. Shnilarly, 
changing font style or font size will not affect the status of the links. 

In the creation of an electronic book, or subsequently, the electronic book may 
undergo aprocess that maps identifying iudex values to each of the first components. These 
index values can then be accessed by software directives that drive the presmtation of 

25 alternative or linked material (e.g., material at one of the second locations) once a selection is 
made. For each electronic book, these index vahiesmay be containedinahidden table tiiat 
maps the idoitifying index vahes of aU first coiiqx>nents with underl^ 
the linked material (the second location). Moreover, each such first component maybe lioked 
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to one or many linked material locations. That is, the first component may be linked to a 
second component, a third conoponent and so on. An example of a hidden table is presented 
below. 



Identiiyiiig 

Index 
Valuc(s) 


Component 
Identifier 


Link 
Number 


Linked Material 
Identifier 


Linked Material 
DescE^)ti(Ui 


Linked Material 
Location (file location / 
file name / 
corresponding index 
value) 


135 


•^Cezzanne" 


1 


More on 
Cezzamic 


Reference material on 
Cezzanne 


Art- 

Encyclopedia.com/Fren 
chArtists/Index Value » 
1 


135 


"Cezzanne" 


2 


Pronunciation 


PronunciatioD of the 
word 


Webstcrs.com/ 
Websters E- 
Dictionary/Index Value 
-56221 


135 


*X>zzanne" 




Audio Gip 


Audio file providing 
condensed Cezzanne's 
biography 


Viewer/ 
Current file/Index Value 
= 199384 


133-135 


"PotchScene 
by Cezzanne" 




Graphic File 


JPEG file presenting 
Cezzanne's PorchScene 
painting 


Viewer/ 
Current file/Index Value 
= 9382 


5673 


"reactivism*' 




Definition 


Definition of the word 


Websters.com/ 
Websters E- 
Dictionary/Index Value 
= 564 


4948- 
4950 


•*Order Little 
Women" 
menu item 


— i 


Book Order 


Order the book Little 
Women 


Discoveryxom/ 
Little Women 
Order/Index Value = 
672 


4949- 
4950 


"Little 
Women" 




Book review 


Review of the book 
"little Women** 


LiteraiyWorks.com/ 
Little Women/Index 
Value = I 


90462 


"Dental 
diseases" 




TOClinkto 
Document Body 


Link fiom Table of 
Contents to desired 
chapter 


Viewer/ 
Current file/Index Value 
= 69980 


1342 


"Dental 
diseases" 




Related 
discussion group 


Access to Web discussion 
group on gum diseases 


NoMoreCavities.cora/ 
Index Value = 1 


572 


"VegicMaster" 




Product drier 


Order the product 
"VegieMastei^ 


HomePuichases.com/ 
KitchenProducts / 
Index Value « 1 


14 


*^esapeake" 




Video 


Video cUp of interview 

wifli J. Michener on 
writing of Chesapeake 


Viewer/ 
Current file/Index Value 
-38677 


14 


"Chesapeake'' 


2 


Nanation 


Audio file - narration of 
Chesapeake by J. 
Kficbener 


Vicwei/ 
Current file/Index Vahie 
-38678 



As shovm in the table, afiist location Xezzamie''basanidea^ 
''Cezanne" has three links. Alinktoasecondlocationistoanart encyclopedia. Alinktoa 
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third location is to an electronic dictionary tiiat provides apronunciation guide. Alinkto a 
fourth location is to an audio file that plays a short biography of the artist Each of the second, 
third and foiurtii locations have their own index valueiinked material location infonnation 
(i.e., the location of second and subsequent components) can include source location, book 

5 name, chapter, page, line, and word as idendfiedbytiieirindex value. The source location will 
provide the delivery system 200 the necessary information to contact the operations center 
250, the Internet web site 279 (see Figure 2) or anotherelectronicdatabaseandrequestthe 
s^ropriale material for retrieval and download if it cmrentiy does not reside on die viewer 266 
or the home system 258. Intiiecasethatthelinkedmaterialresides on the Internet web site 

1 0 279 or on another electronic database, the location information in the hidden table allows the 
operations center 250 or home system 258 to retrieve the desired material from the Intemet 
web site 279 or from the electronic database. 

Ifthesecondcompon^ (H'linkedmaterial, is locatedat the viewer 266, theprocessor 
in the viewer 266 can cause the linked material to be displayed without any communications 

1 5 with an outside source. For example, if the first component is the name "Cezzanne" and the 
Unked material, or second location, is in an electronic dictionary stored in the viewer 266, the 
viewer 266 can display the electronic dictionary entry for Xezzanne." This linked material may 
be displayed full screen, in a picture-in-picture window, or as an overlay, for example. The 
entry can also remain hidden, until a user of the viewer 266 commands the entry to be 

20 displayed. 

Upon selection of a con^onent with underiying links, ftye software directive detmnines 
the idraitifying index values associated witii the selected component, searches flie table for flie 
index values of the selectionmade, looks up the corresponding linked location, accesses the 
location, and displays the linked material on the viewer 266. The linked material can be 
25 displayed on the viewer266 inplaceofdieorigmalsouicematerial,orsimu]taneously with the 
originalsouzcemataialby displayingtfaelinkedmatmdi^ viaa 
split screen, or via a sorem overlay. 
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Figure 20 shows a portion of a page of electronic text having one or more electronic 
links. Thedisplay602mayincludeashowlinksbiitton6()6andahe^butt^ Theshow 
Iinksbutton606maybeusedtodisplayalinkmenu971 (seeFigurel3andFigure21). That 
is, the viewer266 can be commanded, via the show links button 606, to display all components 

S that have underlying links. The components may be displayed in a highlighted mode, in a 
different color, in aunique font, bold or italic typefece, underlined, outlined, or in reverse 
background mode, for example. Tomakeaselectionofacomponenttoviewtheunderlying. 
linked material, the cursor 745 is used to identify the desired selected item. The ball 743 is 
xised to guide the cursor 745 across the page to the desired selected item, and the selection 

10 button is used to make the selection. Altematively,cursormovement for screen navigation can 
be provided via devices such as a fingeipad, mouse, or joystick. Selection can also be made 
by incorporating a touch-sensitive screen into the viewer 266. 

Figure 20 dq)icts the display before commanding ttie viewer 266 to show links. Figure 
21 showsthedispIayoncetherequesthasbeenmadetodisplayaUunderlymg links. Figure 

15 21 shows the link menu 971, ahelp button 612, and a multi-function button, or puU-down 
menu, 614. The multi-function button 614 can be used as aretum button, abide active links 
button, and a restore active links button, for example. Alternately, several additional buttons 
maybeprovided to select these features. Themulti-fimctionbutton614maybeactivewhen 
the show links button 606 has been operated In Figure 2 1 , the components having underlying 

20 links are *TorchScene by Cezzanne** and Xezzanne." The component *TorchScene by 
Cezanne" is shown with one link and the con^onent "Cezzanne" is shown with three links. 
Also shown in Figure 2 1 is the link menu 971 that lists the links, or components, the link 
number and a description of the linked material. For example, the material linked to the 
component *TorchScene by Cezzanne" is a JPEG video file showing the painting. 

25 Once a link is selected, anon-screenr€tumbutton614 allows the subscriber to retum 

from the linked material back to die originally viewed text The show links button 606 (see 
Figure 20) can be displayed on die viewer 266 either at all times that an electronic book is 
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op^ anytime an underlying link exists, or only when conunanded to do so fiom the viewer's 
menu system 851. 

Whra linked material is displayed (for example, in an overlay fiishion) the subscriber 
can command the linked material to be placed in a hidden state by operating the hide active link 

5 button614orbyuseofihemenusy5tem851. Whenanactivelinkishidden^therestoreactive 
linkbutton 614 is displayed. Operation of the restore active link button 6 14 will display the 
. material linked by the active link. . 

Alternatively, the on screen 'Help" menu 887 (see Figure 13) provides access to 
fijrttier assistance when selectmg links. The Help menu 887 is accessed by operation of the on- 

10 screen help button 612. Related link options willbe available for display on the viewer 266 
by selecting the Help menu 887, using the cursor 745 to do so. The '*Help" function allows the 
subscriber to select which specific links to be displayed on-screen. The links when displayed 
inaybesinq>lyhi^ghtedportionsoftext,orte?ctindiffei€n^ All]inksfi>rthedisplayed 
page can be selected to be displayed. Alternatively, by selecting a range of content in an 

1 5 electronic book that may have multiple underlying links, only links associated with the 
components within the selected range will be presented on the viewer 266 for the subscriber 
to select the specific link desired Altematively, a fixed number oflinks may be selected to be 
displayed on the screen at a time. Altematively, only a certain type of link may be selected to 
be displayed. The types oflinks that are available for display may be listed in apop-upmena 

20 Thesubsciibercanchoosefiomthispop-iq^menuwUchofthelinkstod^ Forexample, 
the pop-up menu could list links for a dictionary and links to an Intemet web site. The 
subscriber could choose to display only the dictionary links. Figure 22 depicts the menu screen 
981 used to manage the subscriber's filtering oflinks to view. 

Figure 23 is a logical representation of the components and links for the example first 

25 component "Cezzanne" shown in Figure 20! In Figure 23, the text block (page) 602' 
containing the first componeDt Cezzanne 980 is shown linked to componaits in olha electronic 
files or documients. Afiistliiik981 links C^zanne980toaieferencematerialcon^ 
which is an encyclopedic entry related to the artist A second link 983 lii3ks Cezanne 980 to 
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a dictionary entry 984 that includes apronunciationkey for the artist* sname. A third link 985 
links Cezzanne 980 to an audio clip 986, whichpiovides an audio file desmbing the ardsf s life. 
If the subscriber chooses the audio clip 986, the audio file will immediately begin playing and 
will be broadcast on a speaker in the viewer 266, or the attached television or the attached 

5 personal computer, as applicable. Thesubscribercanstoptheplaybackbyoperatingthebide 
active link button 614. 

The links described above may also fimction in two directions. A biography of 
Cezzanne could include a link to the JPEG file showing PorchScene . The JPEG file 
PorchScene could be one of several still videos of the artist's woric. This JPEG file could be 

10 linked to an appropriate section of the Cezzanne biography. Then, if the subscriber were 
viewing the JPEG file for PorchScene, the subscriber could display the link to the biography 
and, upon activating the link, have displayed that portion of the biography that discusses 
PorchScene . In this example, &e same link is used to display either the JPEG file or text fix>m 
the biography. The same hidden table can be used with the two electronic files (i.e., the 

15 biography and the JPEG file). Alternately, each electronic filemayhaveitsownhidden table. 
In this alternative, the same link may be referenced in each of the hidden tables. 

In the discussion above, each first component is linked to one or more other 
components. However, the other components (i.e., the second and third components, for 
example) may also be linked together. In addition, other components linked to one first 

20 component may be crossed-linked to other components that are linked to a second first 
component (identifying secondary or tertiary cross-links). For exan^le, the JPEG file of 
PorchScene referred to in Figure 2 1 may be cross-linked to the audio file-biography shown 
in Figure 2 1 . Displaying the JPEG file will result in a cross-link being idoitified that links flie 
JPEG file to the audio file. 

25 The cross-linkmay be indicated by highligihting, underlining, outtining, usingaboldor 

an italics type&ce, using a different font, and using different color text If the cross-linked 
materialis selected, thatmaterial will thenbedisplayed. InfteexampledesCTibedabove,the 
JPEG video file that shows the painting PorchScene can be cross-linked with Cezzanne and 
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its identity wouldbe displayed on the viewer 266. The video file couldbe displayed full screen 
or in a picture-in-picture format The video JBle could also be displayed on flie attached 
television 259 or the personal computer 261 (see Figure 2). Finally, the video file couldbe 
printed by sending the video data and aprint command to aprinter 262 attached to the home 

5 system 258 or to the personal computer 261. 

In the table previously showiL, all the links for a number of electronic books were 
provided in one hidden table. In an alternate arrangement, many hidden tables may be 
provided. Forexample, each electronic bookmaybeprovidedwithahidden table. In this 
alternative, the many hidden tables could form a relational database oflinked material. As an 

10 example, several differentelectronicmedicaltextbookscould each beprovidedwithitsown 
hidden table. An electronic ^eralmedical encyclopedia could also be provided with ahidden 
table. Terms that are listed in one of the several medical electronic textbooks could then be 
linked, in arelational &shionto the electronic medical encyclopedia. The several electronic 
medical textbooks could also be relationaUy linked to each other, to on-line databases and to 

1 5 other electronic files. For example, an electronic medical text book could be electronically 
linked to electronic books, databases and o&er electronic files maintained at amedical school's 
library. 

The hidden table (eittier formany electronic books, or mdividually for each electronic 
book) may be provided by the central provider or distributor as an additional feature that is 

20 paid for separately fix>m purchasing an electronic book. The distributor may offer many 
diflfetent levels of service, such as only linking material (components) stored on a viewer, only 
linking material within a particular electronic book, or linking one or more electronic books to 
electronic files outside the home system 258 (e.g., linking an electronic book to adatabase 
maintained by the distributor at an Internet web site). 

25 Downloading the most current links table for an electronic book from the operations 

centa'250ortheMtemetweb site279refi:eshes4e hidden Unks table, that is, anynewl^^ 
that have been generatedby ttie operations center 250, for example, are added to ttie hidden 
linkstable. The cuzrentlinkstablemaybedownloadedin conjunction with download^ 
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electronic book. Alternately, the current links table may be provided periodically by ttie 
operations cent^ 250. 

As noted above, links within electronic books may be self-contained in nature, where 
all the material to be linked to is resident within the same electronic book file. Additionally, 

5 linksmay also beprovidedbetweenmaterialresiding on the viewer 266. Also, links may be 
provided to material fbst currently resides on the home system 258, if separate fiom the view^ 
266. Finally, links may be provided to material that must be accessed through, a 
communications network. For example, material that is ordered or downloaded from the 
operations center 250 or the Internet web site 279 may be linked to the electronic book. On 

10 screen menus can also be supported on the viewer 266 in the form of electronic book files, 
serving as abook or product catalog or service catalog that allows flie subscriber to link to the 
operations center 250 or the hitemet web site 279 to order additional electronic books or 
products and services at any time by selecting the desired component from the text. 

If the content that is to be linked to is currently not available on the electronic book 

15 viewer 266, the viewer266maypromptthe subscriberto decide whettierto: l)retrievetiie 
correspondingmaterial immediately fiom the home system 258, the operations center 250, or 
the Internet web site 279; 2) wait until the next communication opportunity with the home 
system 258 or operations center 250 to retrieve the material; 3) commence direct outside 
communications with another conunumcatiorissystem(e.g.,atel^^ 

20 In one embodiment, the first conq)onents onttie viewCT266 are aTable of Contents 

and List ofFigures for a book. Making a selection fix)m the Table of Contents and List of 
Figures automatically links to and displays the selectedpage witfainAe electronic book file. In 
another anbodiment, the first components on the viewer 266 may be an Ind©c of an electronic 
book. Selecting the desked topic and associatedpage will cause thatpage to be displayed on 

25 . the viewer 266. In yet another embodiment, flie first component is a footnote or endnote. 
When the footnote is selected, the viewer 266 provides a display of material that fiirther 
addresses the reference. In another CTibodiment, the first conqKHient is aword or phrase that 
hasafinlfaerdefinitionorclarificadonassodatedwitiiiit By selecting the first component, the 
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coirespondmg dictionary definition, foreign translation, or glossary entry will be displayed on 
theviewer266. Thedictionaiydefmitionorforeigntranslationniayalsobeprovidedviaan 
audio file. In this embodiment, electronicbooks can bebundled with other standardreference 
material or alternatively, stand-alone reference material like dictionaries or encyclopedias may 

S be accessed firom within multiple electronic book files. 

In another embodiment, the first component is areferoice to another electronic book 
altogether. By selecting the first component, the selected book is diq)layed on the viewer 266. 
In another mibodiment, on-screenmenubuttons willbedisplayedontheviewerscreen602 
that allow for a quick liiik to the Table of Contents, Index, glossary, and other key electronic 

1 0 book sections by simply selecting the item on the viewer screen 602 with the cursor 745 . In 
yet another enabodiment, the selected first component links the subscriber to a book review or 
series ofbook reviews, providing additional information to assist in the decision of selecting a 
new electronic book. In another embodiment, the selected first component is a book title, 
chapter titie, or text in the body of abook that is hnked to an audio file that s^ves as an audio 

1 5 narration of the selection that is played on the viewer 266. In yet another embodiment, the 
selected first component links to a video file (JPEG or MPEG) that can be displayed on the 
viewer screen 602. Another embodiment is a first component that links to textual aimotations 
and notes that have been created by the subscriber. 

One embodiment includes first or subsequmt conq}oneats that are electronic book tities 

20 tiiat, when selected, Unks the subscriber to tiie operations caster 250 or Ifaelntemet web site 
279 to allow for the ordering of the selected book. In a similar embodiment, the selected * 
component consists of aproduct that, when selected, link the subscriber to the operations 
center 250 site or an Internet web site to allow for the ordering of the selected product Lasfly, 
in another embodiment, the selected con^onent is a topic on which there is a link to an 

25 Internet-based discussion group that addresses the topic in more detail. 

Another embodiment is a link to a world watch live server/site. When a link is 
provided to link a word or phrase to a foreign language dictionary, the viewer 266 may display 
aforeignlanguageselectionfeature. Thesubscriberroaythenindicatewfaidilanguagetouse 
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when activatmg the link. For example, an English word or phrase in the electronic book may 
be linked to aFrench, Spanish or Gennan dictionary. The subscribCTmay specify which of 
these foreign language dictionaries to link to. 
DC. Connection To World Watch Live 
5 Theviewer266mayaIsobeusedtodisplayliveorTecordedvideo. Thevideomay 

1^ distributed using ateleconmoturdcationssysten^ Access 
to video may be provided using links within an electmnic b ook or on an electro 
display. 

A. Obtaining Video From Remote Sites, Communicating the Video to a Web 

10 Site, and Streaming the Video To Users. 

Figure 24 shows a syst^ that provides video, audio and text data from remote video 
sources such as videocassettes and television programs. Figure 24 shows remote sites 2 1 02, 
remote cameras 2104, videocassette2106, compression devices 2108, 21 14, digital storage 
device 21 10 and web site 21 12. -As shown inFigure 24, avideo camera2104 is used to film 

15 activity atremotesite2102. As discussed below, numerous video cameras at a single remote 
site may be used to obtain different views, text, and audio (for example, stereophonic) of the 
remote site fix)m different angles and orientations. Also, numerous remote sites, each with its 
own video camera, may used as shown at 2102', 2102" and 2104* and 2104". The video 
cameras film events at the remote sites, and record the events on videocassette 2 1 06 or other 

20 suitable media. 

The recorded information is then transported to a web site 21 12, or to a site in 
communication widi the web site 21 12. As shown in Figure 24, the recorded inforoMtion from 
video tape 2 1 06 is then compressed in compression unit 2108 and stored in digital storage 
media 2110. Many compression algorithms may be used, such as MPEG- 1 , MPEG-2 and 

25 Wavelet Compression systems curr^itly available fit>m The Duck Goip, Xing Technology 
Corp., Indeo, Digital Video Arts, Ltd., VDOnet Corp. and Intel Corp., may be used with the 
system. Thedigitalstoragemediami£^beanyknownstoragedevice,sudiasaharddisk,C^ 
ROM, digital video disc (DVD), digital tape, video file serv^ or odier media. 
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The stored and compressed data may then be provided on a number of streamed 
audio-video outputs 2116 fiom the web site 21 12. This enables many users to access the 
stored video, text, and audio, and allows for one user to receive numerous audio-video signals, 
i.e. split the display into numerous "camera" feeds. 

S In addition to providing streamed audio and video from videocassette, the web site 

21 12may provide audio, text, and video fiomtelevisionchatmels. The television signals are 
received by a conventional television receiver (not shown in Figure 24),. and digitally 
compressed by the compression unit 2114 and fed through the web site 2 1 1 2 to the streamed 
output. It is not noraially necessary to store the television programs in a digital storage unit 

10 (such as the storage unit 21 10), since the audio, text, and video is constantly incoming and 
changing. However, certain segments of broadcast television maybe stored in a storage 
device, such as the digital storage unit 2110, for recall by a user. 

Figure 25 shows anoflier system for siq)plying video, audio and t^ data, where similar 
referencenumeralsindicateitemstfaatconespondtotheitemsshowninFig Thesystem 

1 5 ofFigure 25 uses remote cameras and a communication network to provide remote video to 
the web site. Figure 25 shows remote sites 2102, video cameras 2104, compression unit 
21 18, data communication network 2120, web site 2130, digital storage unit 2132, and 
streamed video 21 16. 

As shown inFigure 25, tihe remote sites 2102 are jBlmedby cameras 2104 (as inFigure 

20 24). However,inthissystem,theou^utsofthecameras2104passthroughacompression 
unit 2118. The compressed audio, text, and video is communicated over the data 
communicationnetwork 2120 to the web site 21 30. The data communicationnetwoik 2120 
maybe any network currently known to one of ordinary skill in the art, such as land-leased 
]ines,satelUte,fiberopticc2d}le,micn)wavelinkoranyothersuit^ Theseandotfaer 

25 delivery systems are described in more detail in Section Vn. 

Other suitable networks maybe cellular networks or pagingnetworks. In apagmg 
network, the cameras21 04 may becormected to apag^ device and/or digital storage media 
or pflgin g tKmiqni tier for communicatjon of the video (including text and still images - as used 
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hereinafier,videoreferstomovmgandsti^ The 
followingpublications, hereby incorporated by reference, disclose relevant systems: PCT 
Publication No. WO 96/07269, published March 7, 1996 by Jambhekar et al.; PCT 
PubUcationNo.W096/21173,publishedJulyn, 1996 by Harris etal.;PCTPubU^^ 

5 WO 96/21205, pubUshed July 11, 1996 by Harris et al. 

The web site 21 30 in this example is adapted to receive information from the data 
communication netw:ork 2 1 20 . The web site may p ass the video fir^ 
at streamed video outputs 2116. Alternatively, the web site may contam a decompressor to 
decompress the video prior to streaming it to users, or change the compression scheme of the 

1 0 video to one which is compatible with the coimected user. Alternatively, the video may be 
compressed at the streamed video output and users who connect to the web site 2130 may run 
decompression software. Theweb site2130may storetheaudioandvideoreceivedoverdata 
communication network 2120 in digital storage unit2132 before providing itto the streamed 
outputs 2116. Altematively, the audio and video may be directly passed to the streamed 

15 outputs 2116. 

Figure 26a shows another embodiment ofthe invention that combines the embodiments 
ofFigures 24 and 25 and adds remote camera control. Figure 26a shows remote sites 2 1 02, 
cameras 2104, computer 2134, video path 2122, 2129, control path 2124, 2126, 2128, 
compressors 2108, 21 14, 21 1 8, 2136 data communication netwodc 2120, web site 2140, 

20 digitalstoragemeaiis2132,andstreamedvideo2116. As with Figures 24 and 25, remote 
sites2102 are fihnedby the cameras 2104. As withFigure 24, theweb site 2140 is able to 
receive data from the video t^e 2 1 06, compress the audio and video in the compression unit 
2108, and store the compressed audio and video in the digital storage unit 2110. Audio and 
video from television stations may also be compressed by the conqiression unit 2114 and 

25 stored or passed as streamed video 21 16, as in Figure 24. 

Likewise, tiie cameras 2 1 04 may be cormected to the compression unit 2 1 1 8 (as in 
Figure 25) and communicate compressed audio and video to the web site 2 140 via the data 
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coinmunicatiQnnetwoik2120. Thusfhefimctioiispeiformedbylhesyste^ 

24 and 25 may be combined in a variety of maimers at the single web site 2140. 

Figures 26a and 26b add the additional feature of camera control to ttie previously 
described systems. As shown in Figure 26a, thecomputer2134is connected to the remote 
cameras 2104. The computer is able to control a mechanical or electrical device on the 
cameras2104, to alterthecameras' orientation (includingpositionand/orangle). Audio and 
video &om the camera21 04 passes to the computer 21 34. The video may be processed and 
stored in the computer. As shown ihFigure 26b, the computermay be connected to multiple 
remote cameras 2104' and 2104" so that multiple users may each control a camera. The 
computer2134may either containacompressororbeconnected to an external compression 
unit 2136. The video from cameras 2104' and 2104" is compressed and provided to data 
communications network 2 120. This compressed video is subsequently recdved by the web 
site 2140. Theremote cameras 2104', 21 04" (Figure 26b) maybe controlledby control signals 
passed from the computer 21 34 on the path 2124. The control signals are receivedby the 
computer 2134 from the data communications network 2120 over the camera control path 
2126. The web site 2140 provides the control information to the data commimications 
network 2120 over path 2128. The web site 2140 in this system is adz^ted to pass control 
signals to the cameras 2 1 04 and to store video images in the digital storage unit 2 1 32. The 
web site 2140 provides a number, of streamed video outputs 2116. 

The system shown inFigure 26aallows remoteusers to control the angle or orientation 
of cameras 2104', 2104". Users are connected to the web site2140 and receive tiie streamed 
video 2116 from the cameras 2 1 04', 2104". If the users wish to move the cameras 2104', 
2104" to the right, they may enter auser command (such as '*panrighf *) at their temiinal. The 
temiinal maybe an electronic book home syston 258, apersonal data apparatus (PDA) or a 
personal computer, for example. The command is received by the web site 2140, and 
formatted, if necessary. Thecommandisoutputtedto&edataconmiunicationnetwoik2120 
as a control signal through die camera control path 2128. The computo: 2134 receives the 
camera control signals from the communication network 2120 over the camera control pafri 
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2 1 26. The remote computer 2134 may be adapted to control multiple cameras at multiple 
locations 2102, or multiple cameras at the same location 2102. 

Whai using the home system 25 8 to coimect to a world watch live web site, the user 
may enter commands using the viewer 266. In this embodiment, the viewer 266 may display 

S commandoptLonsina"softke/'fonnaL The userinitiatesacommand through use of a touch 
screea device or apointing device such as a trackball, for example. The userinterface aspects 
oftheviewer266willbedescribedlaterinmoredetaiL The home system 2S8 and the viewer 
266 may be adapted to display data from the world watch live web site 2140 by the addition 
of software routines. The software routines may be contained on a computer-readable 

10 medium, such as a floppy disk, for example. Alternatively, the software routines may be 
downloaded to the home system 25 8 and the viewer 266 over the Xntemetfiiom the web site 
2140. 

The con3puter2134 is connected to the remote camera2104 by acamera control path 
2 1 24. This path allows control commands from the computer to travel to the cameras 2 1 04', 

15 2104"andcontrolthecameras2104',2104". The cameras 21 04', 2104" may have computer- 
controlled swivel motors (not shown) for panning left and right, may have a computer- 
controlled pivot motor (not shown) for panning up and down, and may have a computer- 
controlled motor (not shown) for moving a2X)om lens. These motors are known to the artisan 
and are currently available. A number of cameras may be provided at a single site to allow 

20 multiple users to have camera control at the same time. 

This system of obtaining and/or storing video at a web site is extremely flexible. The 
system allows forperceived camera control by multiple camoas, actual camoa control of one 
or more camaas, perceived camera control via a wide-angle lens on a single camera, and for 
the generation of comprehensive interactive programs. 

25 B. Perceived Camera Control With Multiple Cameras 

In an alternative system, shownmoreclearly inFigures27 -29,us«:s are given the 
psceptionofcameracontioL Toachievetfais,aplundityoffixedcameras2104,2150,2152, 
2153, 2154, 2156, 2158, 2160, 2162 (Figure 27) are disposed around aremote site 2102. 
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In accordance with this embodiment, it £q)pears to users that fliey are controlling tiie angle or 
position of a camera when in actuality they are merely being transferred to the video output of 
a diflferent camera. Figures 27 - 29 show this concept in greater detail. 

As shown in Figure 27, abuilding 2 1 46 is being prepared for demolition. Disposed 
5 around the building 2146 are cameras 2104, 2150, 2152, 2153, 2154, 2156, 2158, 2160, 
2162, connected to acomputer 2135. The computer 2135 is connected to a communication 
network2120 (not shown). The video fix)m cameras 2104, 2150, 2152, 2153, 2154, 2156, 
2158, 2160, 2162 is digitized and preferably corhpressed prior to communication over 
network 2120, either by compressors coimected to the cameras (not shown) or by a 

10 compressor connected to the computer 2135 (not shown). The cameras may be digital 
cameras or analog cameras connected to an analog-to-digital converter. 

The cameras specifically identified around theperiphery are cameras 2150, 2152, 
21 53, 2154, 2156, 2158, 2160, and2l62. For reference, thebuilding contains the letter "A" 
and the letter '"B" on two sides as shown at 2144 and 2148 in Figures 27 and 28a - 28d. A 

1 5 number of additional cameras 21 04 are disposed about the periphery of the building in a 
circular pattem. The pattem and number of cameras are not critical, but will control how the 
user perceives movement of the "camera". 

Referring to Figure 27, a video camera2150 faces side A, a video camera2152 is 
between sides A and B, a video camera 2153 faces side B and a video camera 2154 is 

20 betweensideB and the side opposite side A. The video cameras 2156, 2158, 2160 and2162 
are disposed closer to the building, as shown. All the video cameras contain audio pickups 
(preferably stereo). Additionally, all the video cameras are connected toacomputer2135 
which outputs compressed audiovisual signals to the communication network 2120 and 
consequently to the web site. The system shown in Figure 27 may be implemented by the 

25 systems shown in either Figure 25 or Figures 26a and 26b. Any number of users in 
communication with the web site 2130, 2140 may receive the audio and video fiom these 
cameras. 
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Figure 2 8a shows a typical screm view 2 1 S 0 of the video presented on the viewer 266 
to remote users who are connected to the web site of the present invention. As shown, the 
user is observing live video fromcamera2150, whichprovides aview of the building onside 
A. A **tooIbaf' of conunands 2 1 5 1 is presented to the user, including apau left connnand 

5 a pan right command a pan up command "t" and a pan down command "l". An 
"autopan" command is used in conjunction with another command (such as panright). The 
"autopan** command is used to autonqiatically move, the picture position in the direction 
previously entered. Forexample, ir*autopan" is entered after ^'panrighV'flien 
keep panning right until another key is pressed or a default key (such as the ESCape key) is 

1 0 pressed. The speed of the "autopan" function is controlled by the "speed" command, which 
is used in conjunction with the and commands. Additionally, the and 
commands, \^en used alone, control a ''zoom-in" and ''zoom-ouf fiinction, respectively. The 
'toolbar" commands are selected via a user input device, which may be a touch sensitive 
screen, a keyboard, mouse, trackball, and a remote control. 

15 Whenany userwishes toswitchfrom the viewofthecamera2150 (Figure 28a) and 

pan to the right, the user initiates apan right command which is transmitted to the web site 
2130, 2140 (Figures 25 and 26a and 26b). The web site receives the command, and in 
response, causes die video fiom the camerapositioned to the right of the camem21 50, in this 
case the video camera21 52 (Figure 27) to be transmitted to the user. The user then observes 

20 thepictureq)pearinginFigure28b,^chq)pearstobeaviewtotherig^tfiom&epre^^ 
position (cameia2150). Iffheusercontinuestopanri^t,theuserispresmtedwiththeFigure 
28c view, received j5omthecamera2153. Tteusermaycontinuetopanri^t all away around 
the building in this manner. 

Additionally the user has special functions available, such as "autopan" and "zoom." 

25 For exanq)le, "autopan" in conjimction with ''pan right*' would cause the view of thebuilding 
to rotate, at a speed dictatedby the "speed" function and the and "-" keys. Using die 
and keys alone causes ttie view to change to a closer camaa ("+") or a camera further 
away ("-''). As shown in Figure 27, tiiie cameras 2156, 2158, 2160 and 2162 are disposed 
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closer to the building than cameras 21S0, 2152, 21S3 and 21S4. A '"magnified" image, 
obtained from the camera 2 1 5 6, is shown in Figmre 28d. If no cameras are disposed closer 
or fiirthCT away, digital ijmage processing may be used to digitally increase or reduce the size 
of the image. The software which controls these functions maybe disposed either at the web 
server or on the user's computer. 

Hius, users may obtain different views of &ebuilding2146 as if they were remotely 
controlling the positioning of a single remote camera. Thexisersmay observe ttie demolition 
of flie building from many exciting perspectives. This **perceived" camera control is 
advantageous because it allows any number of users to "control" a camera. A single camera 
which is remotely controllable is only controllable by a single user. Thus, the present invention 
is suitable for large audiences. The realism of this poroived control is directly dependent upon 
the number of cameras and their distances from the viewed object. 

Therefore, whenthe building 2146 is demolished, any number of users may pan around 
the building m real time as if they were actually present at the site. When the building is 
demolished, flie video cameras pick up, preferably in stereo, the soimds of the demolition. 
Users who have loudspeakers connected to their computer may experience the demolition 
almost as if they were present. 

Figure 29 shows adeployment of anumber of cameras 2104 which are arranged in 
a linear &shion aioimd a point of interest, each camera coimected to computer 2 1 35 as in 
Figure 27. As with Figures 27 and 28a and 28b, this embodimoit uses ^'pareived" camera 
control which may be achieved by the systems shown in Figures 25, 26a and 26b. In this 
example, the remote location and point of interest is a parade, such as a New Year's Day 
Parade. With the camera deployment shown, a user may traverse the length of the parade 
without actually being present. Users may view whichever part of the parade they are 
interested in, for as long as they desire, without worry tibat they havei^^ 
orfloaL Inthisexair^le,thecameradeploymentmerelyfollowstheparaderoute. Parents 
who have children in a band or float may search for the child and follow the diild dirougihout 
the parade route, rather thanhaving to monitor every mom»t of ttieparade on television in^ 
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h(q)esthat1he(^dwiUpass&ereviewingcam^i^m Tbepaxents 
merely ' ve' ' from different cameras along the parade route as their children progress in the 
parade. 

C. Actual Camera Control of Single/Multiple Cameras 
5 Figures 30a and 30b show another system, where anumberof cameras2160, 2162, 

2164,2166, arepiovided. These cameras are in direct commmncation with and are controlled 
by computer2170. Although it is possible to formaringofcamerasto perform •^perceived" 
cam^control (as inFigures 27 - 29), the emboc^ent showiiuses four cameras 2160, 2162, 
2164, 2166 which containmotors 2105 (Figure 30b) for controlling the camera*spositioning. 
10 Themotors are controUedby computer 2170. Eitherasinglecomputa:2170oranumberof 
computers 2 1 70 may be used. The remote location and point of interest shown in Figures 3 Oa 
and 30b are, for exan^le, a watering hole or desert oasis. Users who access the web site 
2 1 40 are able to observe live video of wildlife behavior at the watering hole. The cameras 
2160, 2162, 2164, 2166 are disposed at an island in the middle of the watering hole. The 
IS toolbar 21S1 of Figure 28a is also used in this embodiment and enables users to choose 
rsmyp^ mntrnl rnmrnanHs tn spin the cameras around orperform other camera functions^ such 
as zoom. Users are therefore able to receive different views and angles, and observe tiie «itire 
watting hole. 

Figure 30b shows the control and video paths of the Figure 30a system combined with 
20 system shown in Figures 26aand26b. The video from cameras 2160, 2162, 2164, 2166 is 
conmiunicated toconaputer2170, in compressed oruncompressedform on pafh2122. The 
computer2170communicatesftevideotDconmiunicatioiisnetwork2120forrecq)tionby 
web site 2140 (Figures 26a, 26b). Preferably the video is digitized and compressed by either 
the cameras 2160, 2162, 2164, 2166, the computer 2170, or an external analog-to-digital 
25 convCTter (not shown) and compressor 2136 (Figures 26a, 26b) prior to transfer to the 
communications network 2120. 

C amera control commands are received by the conq)uter 2 1 70 on control line 2126, 
as shown in Figures 26a, 26b and 30b. The commands are formatted, if necessary, by 



wo 01/67302 



PCT/USOl/04946 



68 

computer 2170 and transfenred to control umts 2105 atta^ 

2166. Thecontroliuiits210S are comiected to spm, zoom, or otherwise control the cameras 
as directed by the user. 

Communications links 2124 and2122may be wired, wireless, digital or analog, and 
S computer 2170 may be located nearby or remote from the site 2102. 

The system ofFigures 30a and 30b are unlike the systems shown mFigures 27 - 29, 
because eachuser is assigned aremotecsunerain the Figure 50a, SObemb^ Since 
each user must be assigned their own controllable camera, users will have to contrad for 
available cameras. ThenumberofcontroUablecamerasniayrange&)masinglecamerat^ 
10 number, andispreferably statistically determined to correlate to the average number of users 
who access the web server 2 140 at any given time or at peak times. The number of cameras 
may be reduced by using known systems which utilize queuing, reservations, and time 
limits. 

D . Pew^eived Camera Control Using A Single Camera And A Wide- Angle Lens 
15 Figures 31aand31b show anottier embodiment, usingonlyasingle camera, where an 

unlimited number of users may view any portion of the remote site 2 1 02. This embodiment 
uses a spherical lens 2 1 82 in optical communication with the camera 2 1 80. The remote site 
2102 shown in Figure 31a is a remote watering hole or oasis as in Figures 30a and 30b. 
As shown in Figure 3 1 a, a camera 2 1 80 has a spherical (or otiier wide angle) lens 
20 2182, which provides a 180**, spherical or other wide-angle view. This view, which is 
communicatedtoacomputer2184,containsdistortion. Thecomputer2184commimicates 
and compresses the distorted video back to the web site 2130 or 2140 which stores and may 
process the image. Ratiierthan using the con:q)uter2184, a simple Iransmittermaybe used to 
convey the entire spherical video to the web site 2 1 30, 2 1 40 (Figures 25 and 26a and 26b). 
25 By using 2{px)piialeiinagQprocessing software, tiie web siter^iovestiiebairel distortion and 
storesdatarelatingtotheentiresphCTcalview. Users may then access difierentportions of 
tfael80'' sphere. Ihthis embodiment, the toolbar 21 51 ofFigure 28a is also used. Byusing 
tiie toolbar 21 51, users may move across the spherical view and obtain tiie 'perception" of 
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camera control. This embodiment is advantageous in that it can provide the perception of 
camera control to any number of users simultaneously usiag only one remote camera. 

Figure 3 lb shows altemative embodiments of the system shown in Figure 3 1 a. As 
shown inFigure 3 lb, the spherical (or other wide angle) lens 2182 isused with video camera 

S 180\ ^chconveys video information to computer2184. Computer2184 communicates the 
videoovercommunicationsnetwoik2120tothewebsite2130. Thewebsite2130may store 
orprocess the received video, and make the video available to users atusertemunals 2302, 
2304, 2306, 23 08, 23 1 0 by communicating the video over communication network 2125! 
Communication network 2 125 is explained in more depQx below with respect to Figure 

10 33. 

Because wide angle lenses generate distortion, processing is conducted on the distorted 
image to remove tke distortion from a segment of the image. This processing may be 
performed at the computer 2 1 84, or the web site 2130. Alternatively, the processingmay be 
performed at the user terminals 2302, 2304, 2306, 2308, 2310. 

15 Thus,thewebsite2130hasavailablewideanglevideoforsendingtousers. Users 

displayandviewonlyasegmentofthewideanglevideo atatime. Then,byusingtooIbar2151 
(Figure 28a), the usct may select adjacent segments of the video for view. When auser selects 
an adjacent segment of the video for display, the adjacent segment is processed to remove 
distortion and then displayed Displaying the adj acent segment gives the q>pearance that the 

20 camera was physically '^oved" to the adjacrat side of the original segment. 

One system for electronically removing the distortion &om a segment of an image 
obtained fiom a fish-eye lens is disclosed in U.S. Patent No. 5, 1 85,667, issued February 9, 
1 993 to Zimmerman, incorporated herein by reference. Zimmerman's apparatus iises the 
fbUowinghaniware&rprocessingacqsturedanddigitizedimage: amiCTDConq)ut^counected 

25 to aremote control, computer control, X-Map and Y-M^; aniiq)Utimagebufferconnected 
to the X-M^ and Y-Map with an output connected to an image filter and an ou^ut image 
buffer. This hardware, for example, or any other suitable hardware, may be placed at the 
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computer 21 84, or the web site 2130, but is preferably located at the user terminals 2302, 
2304,2306,2308,2310. 

Alternatively, the specialized hardware is removed and the hardware functionality is 
implemented in soflware at the computer 21 84 or web site 2130, but preferably the software 
is loaded into the userterminal 2302, 2304,2306, 2308, 2310. Thus, inaccordance with ti^^ 
present invention a ^herical (or other wide-angle) image is suppUed to die user' s terminal, 
which executes appropriate software (which may be a "plug-iii" for abrowser application 
program) fordisplayingas^m^of1heimage(orvideo)wi&outdis^ Additionally, Ihe 
distorted spherical image (orvideo) may be saved to astoragemedium, either at the user's 
terminal or at the web site, for ftiture loading and viewing. 

Figure 3 lb also shows how to remove the lens distortion without special processing. 
As shown in Figure 31b, a spherical (or other wide angle) lens 2182 is in optical 
conmiunication with a video camera 2 1 80\ However, a nonlinear imaging sensor 2 1 86 is 
placedbetweeu thespherical 1^2182 and thevideocamera2180\ The imaging sensor is 
designed to provide a distorted output which cancels out the distortion of the spherical lens 
2182, and thus an undistorted wide-angle image is provided to video camera 2180'. 
Alternatively, imaging sensor 21 86 may itself provide a digital output, maldngit unnecessary 
to use a camera 21 80*. hi this case, the imaging sensor 2 1 86 wouldbe directly connected to 
computer 2184. 

£xanQ)les ofimaging sensors 21 86 are disclosedinU.S. PatentNo. 5,489,940, issued 
on February 6, 1 996 to Richardson et al., and in PCT pubUcation WO 96/12862, published 
June 13, 1996toRichardsonetal.,eachincorporatedhereinbyreference. Othersuitable 
imaging sensors may be used with the present invention. 

The image obtainedby the imaging sensor 21 86 maybe undistorted and not require 
fiutherprocessing. Asegmaitoflheimageiiiaydienbeselectedfordisplaybysimplypassing 
fteinu^e data to a display device. Iftheimagingsensorisinq>^ect,furlherprocessingniay 
occurtocorrectfordefectsinthesensor. Additionally, fiirfliCTprocessing for "zoom" and 
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'\mzoom" functions may occur. This furtherprocessing may take place atfhe web site 2130 
or at the user's terminal 2302, 2304, 2306, 2308, 2310. 

The embodiments ofFigures 48a through 5 lb may be used in conjunction with either 
live audio and video or prerecorded video data (with audio) (shown in Figures 24 - 26b) . For 

S example, if nothinginterestingis happening atthewateringhole, aconnectedusermay access 
astored audio and video clip of alion attack which occurred the day before. If "perceived" 
camera control is utilized, the stored audio and video preferably includes all camera angles (or 
a wide-angle view), such that the ability to pan and zoom is preserved. 
E. Web Site Configuration 

1 0 Figures 52a and 52b show a more detailed view of the web site, listed as web site 

2140 (Figure 28a), but whichmay also correspond to web sites 21 12 (Figure 24) and 2130 
(Figure 25). The web site 2140 is connected to a data communication network 2120, the 
Internet 2242, and direct connections 2244. The web site contains transmission equipment 
2210,receive equipment 2220, 2220', two compressionunits 2108, 2114, a web server 2200, 

15 arouter2230, and communication equipment 2240. The web server 2200 itself contains a 
digital matrix switch 2250, aplurality of digital video servers 2252, 2252', 2252", 2252", a 
firewall access control unit 2254, a database server 2256, an audio and video storage unit 
2258, a data storage unit 2260, an administrative unit 2262, a digital matrix switch 2264, a 
camera control unit 2268 and a digital video matrix switch 2270. 

20 The web site 2140 is connected to the data communication network 2120 by 

transmission equipment 22 1 0 and receive equipment 2220. As shown, multiple receivers 
2220, 2220' may be used Also, as shown, the receivers may have more than one video 
output. Audio and video signals may also be input to the web server 2200 by videocassette 
(or other suitable recorded media) or snnply by feeding in televisionprogramming. As with 

25 Figures 24 and 26a, these signals are preferably compressed by compression units 2 1 08, 
2114. On the opposite side, the web server 2200 is connected to remote users by a router 
2230 and communication equipment 2240, which in turn are connected to <he Internet 2242 
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or directly coimected2244 to users. The conimumcations equipment 2240 ou^uts the video 

streams 2116 through a number of input/output ports. 

As previously stated, the web server 2200 contains a digital matrix switch 2250, a 

plurality of digital video servers 2252, 2252', 2252", 2252," a firewall access control unit 2254, 
S a database server 2256, an audio and video storage unit 225 8, a data storage unit 2260, an 

administrative unit 2262, adigitalmatrix switch 2264, a camera control unit 2268 and a video 

matrix switch 2270. . 

The digitalmairix switdi2250receives all incoming compressed video signals fiomthe 

receivers 2220, 2220' and the compressor units 2108, 21 14. The matrix switch 2250 also 
10 receives compressed video data firom database server 2256. Under control of the 

administrative unit 2262, the digitalmairix switch2250 outputs the iiq)ut compressed video 

signals to digital video s^ers 2252, 2252', 22S2"» 2252"*. In this mamier, any iiq>ut signal can 

betransf^edto any video server as directed by the admin unit Also, stored programming 

fromthedatabase server 2256 is routed to the digital matrix switch 2250to be switched as if 
15 it were incoming live video. The outputs of the digital matrix switch 2250 also connect to the 

database server 2256, so that anything at tiie inputs, such as incoming live audio and video, can 

be stored in the database server 2256. 

The compressed input video is passed into various digital video servers 2252, 2252*, 

2252", 2252*" for formatting. Users who connect to web server2200preferably run their own 
20 decompressionsoftwareso thattheno decompression need occur at the web server 2200. 

As an alternative, the digital video servers may decompress the input video. 

The audio and video fiom the video servers 2252 are passed througha second digital 

(video) matrix switch 2270. Since switching has already occunied at the digital matrix sw 

2250, the second video matrix switch 2270 is not required, but is desired for maximinn 
25 flexibility. Itisalsooptiinalwhere1faenumberofusersexceedstheiaimberofvideoiiqf)ut^^ 

one input may be chamieled to numerous coimected users. 

The matrix switch 2270 may contain a processor that j oins diffarent fi:ames of video 

and audio sudi that eadi output contains fi:ames for multiple video pictures including audio). 



wo 01/67302 



PCT/USOl/04946 



73 

This cables users to receive split screen images of video and select an audio track for 
playback (see Figure 37, discussedbelow). The split-screen images maybe formedby using 
knownmethods, whichmay differ depending on the type of compression used. For example, 
digital images n[iay be decompressed, combined with other decompressed images, and then re- 
compressed; or the images maybe decompressed and converted to analog, combined, and 
then converted to digital and compressed for transmission. 

The signals switched by the video matrix switch 2270 are preferably tf^ 
becausethecommunicatedvideostreanis2116arepreferd>lydigi^ Itisprefeiied to process 
all the signals in the web server in the digital domain to improve simplicity and maintain 
maximum flexibility. 

The various streams of video output from the video matrix switch 2270 are passed to 
the firewall access control unit 2254 for output to the router 2230 and the commimication 
equipment 2240. 

Using this system, any usermay receive any signal present at any ix^ul, including stored 
signals within audio and video database 2258or datastorage unit 2260. Additionally, any 
compressed digital signal present at the input to digital matrix switch 2250 may be stored in the 
audio and video storage unit 2258 or data storage unit 2260. This is advantageous in the 
perceived camera control embodiment (Figures 27-3 lb) wh^e the web server 2200 must 
output a difTerent video picture to the user upon user request. When the user request is 
receivedby&ewebsaYQ:2200, the administrativeunit2262 directs themat^ 
and 2270 to output the correct video stream to the user. If the user is requesting stored video, 
the administrative tmit directs the database server 225 6 to provide the video to digital matrix 
switch 2250. If gr^hics or textual data are required, the administrative unit 2262 directs the 
database server 2256 to output the text or gr^hics to digital matrix switch 2264. 

Alfliou^ shown as one functional box, the database server 2256 maybe implemented 
by using several s^ers and/or multiport servers. The audio and video storage unit 2258 and 
data storage unit 2260 may be implem^ited by using niany storage media of diff^^ 
such as optical storage devices (i.e. CD-ROM), magnetic disks, magnetic tape, or memory 
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circuits (i.e. RAM/ROM). The number of units depends ontheamouht of stored data, the 
number of users, and the desired output speed. The database server 2256 may be one or 
multiple units. The audio and video storage unit 225 8 stores (preferably compressed) audio 
andvideopreseatations,mcludingallrelevantcameraangles. The video servos 2252 may also 

5 be implemented as one or more servers and/or multiport servers. 

The data storage unit2260 is used to store informationreladng to audiovisual displ^. 
. ThisinformatipnrelatestothemCT^ustnictuieandscreendispky^ 
users. The stored information may also relate to specifically to the audio and video which is 
cxirrently being displayed and heard. For example, in the demolition embodiment of Figure 

10 28a, a user may click on a *'more info*' icon, to obtain information on demolition. Such 
information, which could include statistics on dynamite, for example, wouldbe stored as text 
or graphics in data storage unit 2260. The ' Wre info" command would be transmitted to the 
communications equq)ment 2240, pass through tiierouter 2230, and the firewall access control 
2254 to administrative unit 2262. The administrative uiiit 2262 then directs the database server 

1 5 225 6 to recall the relevant information, such as statistics on dynamite, fiom data storage device 
2260 and pass the information to digital matrix switch 2264. The recalled information is then 
passed to the firewall access control imit 2254, the router 2230, and the communication 
equipmmt 2240 for transmission to the proper subscriber. The datamay be combined with 
audio and video in the firewall access control unit 2254, or be a separate transmission. 

20 In the perceived camera control embodiment, the communication equipment 2240 

forwards the user's conmiand (such as **pan righf) to the router 2230, which detects the 
command and forwards it to the firewall access control unit 2254, which passes it to the 
administrative unit 2262. The administrative imit 2262 controls the video being fed to ^h 
connected user. Theadministrativeunit2262alsorespoiidstouserconmiandsbyinstructi^ 

25 eitherthematrixswitch2250or tfaematiix switch2270 topass adifferent audiovisual signal 
Scorn another source (i.e. cam^ for example, the camera to the right of the presmt camera) 
to file connected user. If the user is receiving a stored image firom database 2258, the 
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administrative unit instructs the database server 22S6 to recall the appropriate video 
signal. 

In the actual camera control embodiment (shown in Figures 26a and 30a), commands 
fiom the user (such as "pan righfO are received by the coDMnunicationequipmCTLt2240 and 

5 forwarded to the router 2230. The commands enter the web server 2200 via the firewall 
accesscontn>lunit2254,andarepassedtotheadmimstrativeunit2262. The commands may 
be stored in the administrative luut 2262 orpassed to the database server 22^^^ 
the commands pass through the camera control unit 2268 which formats tiie commands as 
necessary for remote camera control The formatted conamands are passed to the transmission 

10 imit2210. Thetransmissionunit2210providestheconiniaudstodatacommunicationnetwoi^ 
2120 for reception at remote cameras and CPU 134 (Figure 26a). 

In the system with the spherical (or other wide angle) l©Qs(sho wn in Figures 31a and 
. 3 lb), where the remote camera xises a spherical lens 2 1 82, the administrative unit 2262 
detemoines vAsich segment or quadrant of the audiovisual ims^e is to be sillied to the user in 

15 response to the user's command The spherical image may be stored in database 2258 prior 
to being output to digital matrix switch 2250. The image is split into a number of sections, 
which when combined form the entire 1 80* sphere. By using suitable image processing 
sofiwaie^ftiedistoitionisremovedorDainimizedineac^ The adminisbative unit 2262, 

in response to a user command, determines which segment of the sphere should be seat to the 

20 user. The administrative uiut then directs the database server 2256 to retrieve 

correct segment to flie digital matrix switch 2250. By controlling the digital matrix switch 2250 
and video matrix switch 2270, the administrative unit 2262 is able to ensure that the user 
receives the correct segment of the spherical image. 

In one sy st^ the endie sphmcal (or other wide angle) video is conomunicated to the 

25 user, and the distortion removed by software at the user's tominaL This minimizes the 
complexity ofthe processing necessary at the web site 2140, and allows the user to store the 
entire spherical (or other wide angle) video. 
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lliecoinmumcationeqii^ment2240maybedesigaedto automaticaUy detemiine the 
maviinimi data rate at which infomiatinn can he transmitted to the cQTWiectediLsers Thedata 
rate depends on the type of connection the web sites has with the user, and the type of 
equipmenttheuserisoperating. The commumcationseqiiipm^t may use the niaxiniinn data 
5 ratepossibleassensedjfronithexiser'scommunications. AltOTiatively, users may enter their 
datarates whenpronq}tedby amenuscreen, as showninFigure 38 and described below. The 
dataratesarelhenstoredincommumcationseqdpme^^ Thecommunicatioiisequ^nnent 
2240 may also congress the video streams prior to transmission using any known conqnession 
algorithm. Additionally^tfaecommunicationsequ^mentnmyiemovevideofiam^^ 

1 0 prior to compression, such that the resulting data rate is reduced to be compatible with the user. 

Figure 32b is identical to Figure 32a, but contains an iiq5ut interface 2225 and an output 
interface 223 5 . The input interface 2225 is used to obtain digital video ftom other sources, 
such as a paging system, cellular system, cable television system, etc. The output inter&ce 
coimects the web site to other communications systems such as paging systems, cellular 

1 5 systems, or cable television systems. In the case where the iiq)ut inter&ce connects to an 
analog system, it contains suitable analog to digital converters (not shown). Also, where the 
ou^ut inter&ce connects to an analog system, it contains suitable digital to analog converters 
(not shown). 

For example, the input interface 2225 may obtain images or video from a paging 
20 system,aiidtheoutputnit^iace222Smaybecoimectedtoapagingsystemtob 

or images to aselectivecallreceiver. In1hisregaid,ttiefo]low]ngpublicalionsareincoiporated 
by reference, each of ^ch relates video/images to selective call receivers: PCT PubUcadon 
No. WO 96/07269, published March 7, 1 996 by Jambhekar et al., PCT PublicationNo. WO 
. 96/21173,pubUshed July 11, 1996byHairisetal.,andPCTPubUcationNo. WO96/21205, 
25 published July 1 1, 1996 by Harris et al. 

F. Communication To Electronic Book Home Systems 
Figure 33 shows how the users are connected to the web site usinghome system 258, 
and shows an exan[q)le of a communications network 2125 (Figure 31b) in detail. The 
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connections shown inFigure 33 spply to flie web sitesof ftieprevious figures, inchidmg the web 
site 21 12 (Figure 24), 2130 (Figure 25) and 2140 (Figures 26a and 32a). Figure 33 shows 
a SCTverplatform 2200, the Internet 2242, two direct connection 2244, two traditional Internet 
hosts 2272, 2274, two cable Internet hosts 2276, 2278, asatellite-basedlntemethost 2280, 

S a telephone dialup 2282, anISDN channel 2284, a cable plant 2286, 2288, asatellite system 
2290 and aplurality of connected userterminals. TheusertenninaIsshowninFigure33 are 
homesystenis258includingelectronicbookviewers266. However, the userterminals can 
be any electronic device capable of receiving and displaying digital data includingpersonal 
computers, for example. 

10 In operation, the web sites 21 12, 2130, 2140 may communicate over the Internet 

2242 to a number of different systems. These systems include the traditional Internet hosts 
2272, 2274, the cable headend Internet host 2276 and flie wireless iitemet host 2277. The 
traditionallntemet host 2272, 2274 maybe connected using atelephone line 2282 or an ISDN 
channel 2284 to a plurality of home system 258. The cable Internet host 2276 may be 

15 connected using a cable plant 2286 to a home system 258. 

Alternatively, the web site is connected using a direct coimection 2244 to a cable 
headendlntemethost2278orsatellite-basedlntemethost2280. The cable headendlntemet 
host2278commimicatestoacableplant2288andahomesystem258. The satellite-based 
Internet host 2280 communicates using a satellite 2290 to ahome system 258. These direct 

20 coimections 2244 enable a higher data rate and use a high speed cable mod^. 

The communications equipment 2240 (Figure 32a) may enable communications with 
any type of user terminal, including the home system 258, no matter what the data rate or 
system. User terminals with higher data rates will receive higher quality audio and video 
images. 

25 G. Screen Displays and Features 

Figures 34 - 39 show e?camples of display ps^es that are shown at the home system 
258. The pages and menus are stored in data storage unit 2260 (Figure 32a) as graphical 
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and/ortextualinfbmiatLon. TheworldwatehHvemeaumay beaccessedusinglhemmusys^ 
858 shown in Figure 13. 

Figure 34 shows an example of a home page 2400 that contains a number of 
advertisements 2402, numerous web Unks 2404, a society link 2406, options for viewing 
S televisionpiogramnung 2408, aplurality of rapid access entcy options 2409 including a world 
watch live option 2410, and options for clubs 2412. 

The advertisements 2402 are us eM for the page provider to generate reve^ 
describedpreviously, the system is designed such ttiat television programming can be supplied 
over the Internet. Users may view television programming by selecting the home page 
1 0 television option 2408. The Magazines 2404 are used to provide information concerning 
specific topics to the user. Users mayjoin a society, having additional membership benefits, 
through the "society** selection 2406. The world watch live feature 24 1 0, part of the rapid 
access entry options 2409, is selected when users wish to watch live video fiom remote sites. 
The clubs shown in the club option 2412 are selectedby users who wish to obtain information 
1 5 related to common areas of interest. 

Figure35 shows a society menu 2406, selected from the Figure 34 home menupage. 
As shown inFigure 35, there are options for world watch Hve 2420, is an advertisement 2402, 
subscription information 2424, and numerous club options 2422. Tliis screen and all the 
fimctions selected inresponse to the displayed options maybe provided on asubscription or 
20 temporarily fi^ee basis. 

Figure 36 shows one example of a world watch Uvemeriu2440. Thismenumaybe 
used to select remote locations firom which to observe Uve or prerecorded video. In this 
example, amap of the world is presented with sites that are available to select for observing 
livevideo. Thescreaiindicatessitesthatareactive2442orunderconstruction2444. This 
25 meavL also contains two advertisements 2402. 

TheworldwatchUveembodimentaUowscozmecteduseistovisitvirtually anyplace 
in the world to leam more about its culture, geography, or environment Coupled with 
perceived or actual camera control and associated prestored video, textual and graphical 
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infonnation, apowerful and inexpensive learning tool is realized. This is more closely shown 
in Figure 37, 

Figure 37 shows amenu 2450 that corresponds to the Egyptian siteinFigure 36. This 
screen concerns "Giza, Egypt", and contains live video fipom five cameras. As shown in the 

5 screen,thereiscameraone2452,canierastwothroughfive2454,a*Map"option2456^ an 
"About This Site" option2458, an "About Egypt" option2460, an*TJpcomingEvmts" option 
2462 and a "Remote Control" option 2464. Camera one 2452 is the default for the mam 
viewing camera. The user may select video image sizes and the number of images to be 
displayed, limited by theequipmenttheuseris opiating Video fiom cameras two tibroughfive 

10 aiesuppHedaloiigwitiithatfixsmcameraonetopiovidealteniativesitesand about 
the topic of the screen (i.e., Egypt). 

In an embodiment, the information presented in Figure 37 may be accessed by 
activating a link in an electronic book. The link may exist in an electronic book devoted to the 
history and geography ofEgypt. The link may exist in an electronic encyclopedia. The link 

1 5 may exist as a text address (e.g., aHTML address). The link may also exist as a location or 
an icon in the electronic history book or the electronic encyclopedia. When the link is 
activated, video feed, either live or delayed, may be provided to the user's terminal (e.g., the 
viewer 266, fliepersonalconq)uter 261, orthe television259, shown in Figure 2). The video 
may be accompanied by audio data, text data and geographical data. 

20 The **Map" option 2456 brings theusarbackto the worldmap (Figure 36) to select 

additional sites. The "About This Site" option 2458 brings up text, gr^hics or additional video 
concerning the site of Giza, Egypt. For example, apiofessor q)pears and talks about tiie origin 
of the SphinK (shown by camera 1). The embodiment shown in Figure 39 and described 
below (mtOTctive lecture) naaybe combined with the "About Th^ Additionally, 

25 ofhervideomaybedisplayedinresponseto selection of"About This Site". Such video may 
be a documentaiy of the Spfamx or discussion about the technology that historians estimate was 
used to construct the Sphinx. 
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Hie ''About Egypf optLQii2460bnngs gcapUcs^ text or additional video concerning 
Egypt Forexaniple,amapofEgyptwi1hpopiilaliondensitiesniay be shown. Theoptionfor 
'TJpcoming Events" 2462 brings graphics, text or video concerning new events in Egypt For 
example, text and newspaper articles concerning the construction of new inigation canals is 
displayed. 

••Remote Control" option 2464 brings up a command menu (such as the •toolbar" 
21 S 1 ofFigures 28a- 28d) that allows the user to change camera angles orpositioning in aqy 
of flie cameras capable of that ejQEect. The menu would apply to actual or perceived camera 
control. For example, the user could pan around the Sphinx (camera 1 , shown at 2452) to 
observe it from the front, each side, and back. 

Thus, this single screen relating to Egypt provides a wealth of information at a single 
Intemet address (or web site). Theusermay, but need not,**link'* to other locations on the 
Int^et Audiovisual presentations are displayed, which give tiie user insight into the people 
and culture ofEgypt. Text, gr^hics, and additional stored video is available to fiirther educate 
the user. Camera control (actual or pCTceived) gives the user the feeling of walking around 
different locations in Egypt. 

Figure 38 shows a screen 2470 that asks users about their equipment in order to 
determine the appropriate datarate for coinmuiucatioris. Preferably tiie screen is not needed 
and the data rate is detennined by communication equipment 2240 automatically. Note that 
an advertisement 2402 is also shown on this scre^ 

Figure 39 shows an interactive lecture system. AsshowninFigure39,]ivevideo2500 
of an astronomy professor's lecture is transmitted to coimected users. The users are able to 
ask the professor questions 25 10 andreceive answers 25 1 2. The live video 2500, questions 
2510, and answers 25 1 2 are shown to all connected users . The users may enter questions via 
keyboard or microphone. However, if suitable data rates are available, tiie user may ask a 
questionviavideo. ThusaspUtscreravideoshowingbothtiiepersonaskingfiiequestionand 
thelecturearmaybepresentedto-alluserssimultaneously. The answers arepreferablygivenby 
the lecturer, who may observe the question on aremote display. Alternatively, the answers 
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may be supplied by flie web site as text, graphics, or prestored video. The answermay pass 
throu^ a closed cs^odng device, be encoded, aad displayed on the screen in an answer box 
2512. Othertechniques for two-way video comniiinication are described in co-pending U.S. 
Patent Application Serial No. 09/391,461 entitled VIDEO CONFERENCING USING AN 
HLECTRONIC BOOK VIEWER, ffled September 8, 1999, the.disclosure of whichis hereby 
incorporated by reference. 

Referring to Figure 32a, questions are sent to the web site2140 aspart of the normal 
user temiinal communication. Thewd) site 2140 receives the question atthe communications 
equipment 2240 and forwards the questiontfarough router 2230 acdthe firewall/ access control 
unit 2254 to the administrative unit 2262. The administrative unit 2262 detemaines whether the 
question can be answered by playing stored video or showing stored text or graphics. If so, 
the administrative unit 2262 directs the database server 2256 to recall the appropriate 
information. The information is then output through the matrix switches 2250, 2270 or 2264, 
unda: control oftheadzninistnitive unit, as ^propiiate. The ability ofthe administrative unit to 
answer questions dep^ds upon the complexity of its software. Simple, prestored answers to 
frequently asked or standard questions may be provided in a basic system. More advanced 
systems may utilize an interpreter to analyze the questionbefore providing an answer. For 
exanople, fiequentiy asked questions in the astronomy fieldmay be 'Svhatis a star''? or 'liow 
was the galaxy fonned'7 iiresponseto thesequestions, whichmaybeprovidedonamenu 
or list, the administrative unit recalls prestored answers in eitiier video, text, or 
graphics. 

If a question cannot be answered by the administrative unit, or is scat directiy to the 
remote lecturo:, tfie question proceeds to the remote lecturer in a similar fiishion as the camera 
control signal (Figure 26a) discussed previously. However, in the interactive lecture 
embodiment, the camera control unit 2268 (F igure 3 2a) is replaced with a question format unit 
(not shown) which reformats the question under control ofthe administrative unit 2262. 
Transmitter 2210 thm transmits aquestion signal to the location of the remote lecture via the 
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data cQi]mumcationnetwo±2120 and the cx)mixu^ 2126, 2128. The lecturer has 

a display which shows questions received over fbe data conununication aetwoik. 

In an alternative system, the lecturer or anumber of assistants may select fifom among 
many prestored answers in response to a questioiL In this system, the remote lecturer has a 
S compute and monitor (not shown) which displays the questions and the available prestored 
answers. The lecturer or assistants then match aiiswers with the questions. Theprestored 
answers are preferably forwarded to the individual who asked the associated question. In 
Older for others to learn from the questions, the questions and answers may be provided to all 
connected users. 

10 Figures 40 and 41 showasystemthatusesacombinationoflivevideo, storedvideo, 

stored graphics, camera control and interactive questioning. The live video 2550 of camera 
1 showninFigure40relatestoageologicalsite,i.e.,thegeyser,"OldFaith&l." Sincethesite 
is located on aNationalPark, the display screen has been customized to allow for the selection 
''About NationalParks" 2604. When this is selected, the user's command is communicated 

15 to the web server 21 12, 2130, 2140 for analysis by the administrative unit 2262. The 
Administrative unit 2262 determines that prestored video and gr^hics are required, and 
instructs the database server 2256 to output the correct information: video to the matrix switch 
2250, and graphics to flie matrix switch 2264. The matrix switches, 2250, 2270, and 2264, 
under control of the administrative unit 2262, forward the video and graphics to the user 

20 through the communication equipment 2240. 

Figure41 showstheresultattheuserterminal. The communicated prestored video 
2560 of a Park Ranger appears on the screen. The Park Ranger discusses the topic of 
National Parks. The discussion occurs in conjunction with agraphical display of the locations 
of all National Parks, shown at the screen location 2570. 

25 The usCT may select other options, such as ""Map 600" to retum to the map of all 

remote sites, "About This Site" 2602 to leammore about the site currratly viewed, ''More 
About National Paries" 2614 for even more information aboutNational Parks, 'Upcoming 
Events" 2606 for a schedule of upcoming events, *^emote Control" 2608 for remote (either 
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actual orperceived) control of the camera (i.e. camCTa 1), "Ask Questions" 2610 for asking 
questions (as inFigure 39) to an on-line PaikRanger, and "OtherTopics" 2612, for alist of 
other topics and/or options. 

H. Surveillance Systems 

The zemote camera systems discussed above may be used in a surveillance or tracldng 
system. For exan^le, areseaichermayplacea video camerainifaecenta'of a watering hole, 
preferably coimectedto avideo recorder for storing many hours of activity at the watering hole, 
Multiple cameras or a wide-angle lens may be used such that virtual camera control (as 
described previously) may be performed on the video. Such a surveillance systemhas many 
advantages. 

First, the surveillance system allows for automatic scanning of the surveyed area, 
withoutflieneedformovingany cameras. Additionally, viewingnoultiple segments of the area 
imdersurveillancemaybeviewedattiiesametimeinaspUt-screenormulti-sc^ All 
that needs to be done is the removal of distortion in multiple segments of the video (if using a 
wide-angle lens). ThedisclosureofU.S.PatentNo.5,359,363,issuedOctober25, 1994to 
Kuban et al., incorporated herein by reference, discloses one example usable with the 
surveillance system. 

Second, automatic monitoring and/or trackingniaybeperformed. Often, researchers 
and photographers wait through long periods of inactivity before adesired event occurs. For 
example, a photographer may wait for hours for a lion or oth^ wildlife to approach the 
photographer'sposition. Thesystminiaybeusedtoautomaticallymonitoraremoteregion 
for activity. In this case, aprocessormaymonitorthemultiplecamerasorthe digital wide- 
angle video for pixel changes indicating the desired event. For example, an Eqpproaching lion 
in an otherwise inactive desert environment will cause amovingpattem to form on a camera's 
output or in the wide angle image. A processor may detect the pattern and alert a wildlife 
researcher that an event is occurring. 

Furlh^, the processor may automatically and continually display die relevant camera 
ou^ut, or the segment of die wide angle image containing the Uon, thereby tracking die lion. 
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Thus, the preseat invention may employ tracking techniques, known in the prior art, to the 
obtained digital image. 

In the monitoring and tracking embodiment distortionmay be removed from the wide 
angle image prior to peifoiming the processing to detomine whether an event is occurring. The 
type of event being monitored and nature of the object being tracked controls whether 
moiiitoringand/ortrackingmaybeperfonnedonthedistortedorundistortedimage. Oneof 
ordinary skill in the art will choos e the system best suited for the particular monitored event or 
tracked object. 

Figure 42 shows a flow diagram of a monitoring and tracking system. The software 
necessary to perform the monitoring/tracking functions may be located at the web site or at the 
user's terminal, suchas the home system2S8 and viewer 266. The image/video signal to be 
processed for monitoring and/or tracking may be a live video feed or be played back from 
stored video. Thus» awildlifescientistmay leave multiple video cameras rumiingovemi^t (or 
a single video camera with a wide-angle lens) and when tide video tspe is played back, the 
segments/cameras containing activity are displayed. 

Referring to Figure 42, an 'Input frame ofreference" routine 2700 is executed. This 
routine is optional, and is used to establish a frame of reference direction, such as north. The 
frame of ref^rrace may determine the first segment of a wide-angle image to view, or the first 
camera to view. a "reset segment counter^' routine 2710 is executed. This sets the 
segment or camera to be first displayed. 

Each segment or camera is viewed only for a limited time, prior to viewing the next 
segment or camera. Thus, a "reset timef ' routine 27 1 5 is executed to reset the interval when 
segments or cameras are switched. 

Next, the "obtain image" routine 2720 is executed Tbisroutineobtainsthewideangle 
image (live or pterecoided), or images firom all the cameras (in fiie multiple camerapercdved 
control embodiment ofFigures 27 28d). The obtained hnage &om a wide-angle lens may be 
processed to remove the distortion or not, depending on what is being monitored. 
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The obtained image isprocessed to deteimine active areas (cametas or segments). 
Active areas areareas where the processor detennines that activity is talcing place, either by 
changes in the pixels at those locations, by using other known image/video processing 
techniques, or by using external sensors. Theprocessingisperfonnedasknownintheartand 

5 isnotdescnbedfiirther herein. Ilieprocessing occurs diningthe '"process for activity^'routme 
2730. Thisroutmeuseslhe&ameofreferencetodeterminewMchsegmen^^^^ 
normal (i.e., north) is/are active. 

If activity is present, the "display active segments" routine 2750 displays the active 
segments or cameras on a display. Distortion from the relevant segments is removed in the 

1 0 wide-angle lens OTibodiment. If more than one segment is active, a split screen display may 
show eachsegment simultaneously. The spht screen (Uspiaymayroa^ 
ofreferencethatwaspreviouslyenteredduringroutine2700. The**resettimef'routine2710 
is then executed so that the last segment under view is returned when activity is no longer 
present. 

15 If activityisnotpresent, the "display currentsegment" routine 2760is executed This 

routine displays the current segment or camera until the timer expires, at which point the next 
segment or camera is displayed The display may make reference to the frame of reference 
which was previously entered during routine 2700. 

Afto: displaying the cun:entsegmmt or camera, the '"time limit 

20 isexecuted Ifthetimelimithasnotbeenexceeded,abranchtothe"obtainunage"routin^ 
2720 occurs andprocessmg continues until the time hmit is exceeded, or until activity occurs. 
In an "autopan" embodiment (Figure 28a), the time limit value may be inareased by pressing 
the "-"button in conjunctionwith the "speed" button (Figure 28a), for a slower autopan, and 
the time limit may be decreased by pressing the "+" button in conjunction with the "speed" 

25 button (Figure 28a) for a &ster autopan. 

If the time limit is exceeded, the segment (or cam^) counter is incremeoted by the 
'1naemeQtsegm^counter"routine2780. If&ecounterisgreat^lhantfaeniaximumnmnber 
of cameras or segments, the "counter > max" routine 2790 branches to the "reset segmmt 
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counter" routine 2710, to restart the automatic panning. If the counter is not greater than 
allowed, a branch occurs to the '"teset timef ' routine 27 1 S so &at the next segment or camera 
may be displayed, and processing for activity continues. 

Thus, the flow chart of Figure 42 allows for automatic panning and for automatic 
tracking. If flie "process foractivit/'routine 2730, the "activity?" test 2740, and the "display 
active segments" routine 2750 were removed, &e "autopan" function described previously and 
shown with respect to Figure 28a - 28d would be achieved. .. In this case, "display current 
• segment" routine 2760 would follow "obtain image" routine 2740. 

Monitoring and automatic panning may be combined. When combined, all active 
segments or cameras are automatically panned for a brief timeframe. Thus, if a lion and zebra 
are both moving towards the camera from opposite direction, each would be displayed for a 
brief timeframe before switching to a display of the other. This is an alternative to the split 
screen display previously described. 

I. Display of Video Data 

In the systems described above, tiie user may select or be provided data concerning 
the video currentiy displayed. For example, siq)erimposed on the video may be the date and 
time the video was recorded, a name of the image location, remaining time for the video, or 
data pertaining to the segment (or cam^ source) of the video which is cuireotly being viewed 

This segm^t/bamera data may be acompass heading (such as north) or angle from a 
refermce (such as 40 degrees), or coordinate information (such as X/Y, X/Y/Z, R/0, and 
X/R/0)ielatingto the location of the centerofflie segment/video currently displayed inrelation 
to flie wide angle image or other cameras. A gr^hical representation of the lens (or layout of 
thecameras)may show which segment ofthe wide angleimage(or camera) is being displayed. 
In order to display the image segment, a frame of reference may be adopted, especially for a 
sphmcallens. The frame ofreference would be either generated by aprocessorattiieweb 
site or user's tenninal, or entered by a user or operator. For example, tiie user may select 
wfaidi direction is "nortii" or position the axis of a coordinate system if a coordinate display is 
to be used for a particular lens. 
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Additionally, flie image* s magnificatioxi and its density/colors may also be shown on the 
display, such as •'magnification = lOx, picture density = 200x200 pixels, 64 colors." 

The display of image datamay beused in all embodiments of the present invention, and 
are preferably updated when the displayed image changes. 

Figure 43 shows a display 2800 showing a coral reef 2805 where users have virtual 
camera control using multiple underwater cameras. On the screen 2807, the date 28 1 0 is 
displayed along with the time 2820. The location is shown at 2830 and the rmiaioing time of 
fheprogi:amat2840. The magnification is shown at 2850 and the density and colors at2860. 
The segment camera field 2870 shows that the user is viewing camera no. 3. This 
segment/camera datamay be shown graphically, as depicted at 2880. Field2880 is a top view 
of the coral reef 2805 and the layout of the cameras, in this case cameras 1 through 1 0. The 
square around camerano. 3 indicates that this camera is the source oflhepicture on the display 
2800. The firame of reference (north) is indicated at 2890 for the graphical segment data and 
2895 for the video data. 

J. Storing Video and Interactive Presentations. 

The images, video, and image data may also be stored at the user's temainal The wide 
angle distorted image may be stored, along with the image data, if present. Storage of the 
image and image data enables the usct to retrieve the image and view a segment at a later date. 
Optionally, the entire interactive presentationnoay be stored at tiie user's terminal (including 
associated graphics, tex^ >adeo, data, or other information), although all theperdnmt files and 
data would have to be received by the user. 

ThedisclosureofPCTPublicationNo. WO 96/08 105, pubKshedMarch 14, 1996by 
Labun,incorporatedhereinbyreference is related to storing images and may be used with the 
present invention. 

The video or image may be stored in eitiier its distorted or undistorted state. Storing 
the video or iuGiageinitsundistortedstatehas the advantage in thattaUa^ 
be stored intiieir most viewable state, andintiiateditingmaybeperformedon&^ 
easily if they are retrieved with tiie distortion removed. 



wo 01/67302 



PCTAJSOl/04946 



88 

The artisaa of Qidinaiy sidU wiU ^re^ 
patents and publications incorporated herein by reference may be applied to the present 
invCTtion. As such, tiiepatent plications, patents and publications are incorporated herein 
in tiieir entirety. Thetennsanddescrq)tionsusedhereinaresetfQrlhbywayofillust^ 
and aienotmeant as limitations. Those skilled in the art >vill recognize that numerous vadadons 
are possible within the spirit and scope of the invention as defined in the following 
claims. 
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CLAIMS: 

. 1. An q)paratus for coimnuincatmg audio and video sig^ 
apparatus comprising: 

a web site, connected to a plurality of electronic book viewers, comprising: 
5 a receiver that receives digitally compressed audio and video; 

an audio-video server that provides a plurality of digital video signals; 
a switch tibat switches and combines the plurafity of (tigiM video sig^ 
an administrative unit, connected to the switch that directs which signals are 
switched and combined; and 
10 atransmitter, connected to the switch that communicates the digital audio and 

video signals to the electronic book viewer as video streams. 

2. The apparatus of claim 1, wherein tiie web site further comprises: 

an audio and video storage device, connected to the receiver that stores at 
least some received audio and video; 
15 a data storage device, connected to the receiver that stores textual and 

graphical data; 

a database server, connected to the administrative unit, the switch, the audio 
and video storage device and the data storage device; 

wherein the administrative means directs the database server to retrieve andsiq)ply to 
20 the switch the audio and video infonnati on bom the audio and video storage device and the 
textual and graphical data from the data storage device. 



3 . An £q)paratus providing the perception of rranote camera control to a user, the apparatus 
comprising: 

a plurality of remote video cameras arranged to film a remote site, each r^ote video 
25 camera providing a video signal of a different perspective of the remote site; 
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a compressor, connected to the video cameras, that compresses the video 

signals; 

a data commimication network, connected to a web site and to the compressor, that 
carries the compressed video signals from the remote site to the web site; 
5 the web site connected to thecommunicationnetwoxk and to electronic book viewers, 

comprising: 

a receiver, having the compressed video signals as its input; 

an adnainistrative unit that detemtiines which video signals to pass to a user 
terminal in response to a user command; 
10 a switch, controlled by the administrative unit, for switching received video 

signals to communication equipment; 

communication equipment, in operativecommum administrative 
unit, for transmitting the switched video signals to user temoinals andforieceiving auser 
conmiand; 

IS whereinan electronic bookviewertransmitstheiisercommandtothewebsiteandthe 

adnMnistrative unit directs the switch to provide video signals to the electronic book viewer in 
accordance with the Viser command fliereby enabUng a usct to remotely control the position or 
orientation of the video signal being received at the electronic book viewer by entering user 
commands. 

20 

4. A sjrstem for providing a user with perceived camera control via an Internet web site, 
comprising: 

communications equipmmt to receive camera control commands from one or 
more connected users and to transmit video to the one or more connected users; 
25 electronic book viewers to receive the transmitted video and to transmit 

camera control commands; 

video of diSorrat views of a remote site; 
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an administrative unit, wherein the administrative unit determines which view 
of the remote site to transmit to a connected user in response to a received camera 
control command, therd)y providing the connected user with die perception of camera 
control 

5 5. Thesystemof claim4, wh«:einthe system further comprises a video storage unit, 
wherein the video storage unit supplies video of diffoent views of the remote site to the web 
site. 

6. The systCTi of claim 4, wherein the video of different views of theranote site is video 
of diff^ent camera angles of the remote site. 



10 7. The system of claim 4, wherein the video of different views of the remote site is a 
distorted wide angle video of tiie remote site, and wherein the system further comprises a 
means for removing distortion from at least one view of the wide angle video. 

8. A system for providing a user with actual camera control, the system comprising: 
a web site, comprising: 

1 S communications equipment to lecdve camoa control commands from one or 

more connected users and to transmit video to the one or more cormected users; 
a video receiver for receiving video from a remote video camera; 
a transmitter in communication with the remote video camera; 
a camera control unit, which outputs forpiatted camera control commands to 
20 the transmitt^, and 

wherein the camm control conunands received from a user are formatted and 
transmitted to the remote camera and control the remote camera. 
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9. A method of remotely viewing a remote site, the method con^rising the steps 
of: 

accessing a communications network; 

receiving video depicting one or more views of the remote site via the communications 
5 network; 

entering commands regarding a different view pf the remote site; 
displaying the different view of the remote site. 

10. Tliemethodofclaim9, wherein&econmunicationsnetwoikis&el^^ andfurther 
comprising the steps of: 

1 0 addressing a web site oh the Internet; 

selecting a remote site. 

11. Themethodof claim 9, wherein thereceived video isdistortedwideaaglevideo, and 
wherein flie step of displaying con5)rises the step of rmioving distortion from a segment of the 
distorted wide angle video pertaining to the different view to be displayed. 

15 12. Themethodof claim 9, wherein the received video is video fiomoneof aplurality of 
remote cameras, and further comprising the steps of: 

processing the entered command to select one of the remote cameras in accordance 
with the commanded different view; and 

receiving video of the different view from the selected remote camera. 

20 13. The method of claim 12, wherein the displaying step further includes the step of 
indicatingthelocationoftheselectedremote camera andaframeofre&rence at theremote 
site. 
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14. The method of claim 13, wh^inthestepofindicatingfuithCTCompriseslfaestepof 
graphically displaying a layout of cameras at the remote site with respect to the firame of 
reference. 

1 5 . The method of claim 9, wherein the displaying step further includes the step of 
S indicating the location of a frame of reference at the remote site. 

16. The method of claim 9, wherein the displaying step fui& 

mdicating data conceiimiglhevideo^tfaedataselectedfcom the gm remotesite 
location, remote site time. 

1 7. The method of claim 9, wherein Qie displaying step further includes the step of 
10 indicating data conceming the video, the data selected from the group consisting of: 

magnification, pixel density of the video, nimiber of colors in the video. 

1 8. The method of claim 9, wherein the entered command is a command to monitor the 
remote site, the method further comprising the steps of: 

processing the video for activity at the remote site; 
15 and wherein the step of displaying includes the step of: 

selecting views of the remote site displaying activity if activity is present 

19. The method of claim 18, wherein the step of displaying further includes the 
step of: 

automatically panning ttie remote site if activity is not present. 

20 20. Tbemediodofclaiml8,wfaaeintfaereceivedvideoiswideangledistort^ 

the stq> of processing includes the step of removing distortion from at least a portion of ttie 
received video to detect whether activity is present. 
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2 1 . The method of claim 20, wherein the step of selecting includes the stqp of choosing 
segments of the wide angle video for viewing, and the step of displaying furfh^ includes the 
step of removing distortion fi?om the chosen segments. 

22. The method of claim 1 8, wherein the received video is video from a plurality of 
S cameras, and the step of selecting includes the step of choosing one or more cameras for 

viewing if activity is present. 

23. Themethod of claim 18, wherein flieenteced commandis acommand to automatically 
pantheranote site, and wherein the step of displaying further includes Ihe step of incrementally 
viewing, for a fixed time, a plurality of different views of the remote site. 

10 24. The method of claim 23, further comprising the step of: 
selecting whether to increase or decrease the fixed time. 

25. The method of clam 9, further comprising the steps of: 
receiving data and graphics concerning the remote site; 

and where the step of displaying further conq)rises the step of showing the data and 
IS graphics. 

26. ThemethodofcIaim2S,furthercomprisingthestq>ofsaving1hevi 
data in a storage media. 

27. Themethodof claim9, finlhercomprismgtfaestq) of saving the video, gr^hics, and 
data in a storage media. 

20 28. An electronic book, comprising: 
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alinkto anihtemet web site, the web site providing aplurality of streaming video, 
audio and text data when connected to the electronic boolq and 

a control function, wherein the control fimction allows selection of one or more of the 
pluraUty of str^ming video, audio and text data, and wherein the selected data are displayed 
5 with display of the electronic book. 

29. The electronic book of claim 28, wherein the electronic book is adapted to be 
displayed on an electronic book viewer. 

30. The electronic book of claim 28, wherein the electronic book is adapted to be 
displayed on a television. 

10 31. The electronic book of claim 28, wherein the electronic book is adapted to be 
displayed on a personal computer. 

32. The electronic book of claim 28, wherein the electronic book is adapted to be 
displayed on a palm-sized viewer. 

33. The electronic book of claim 28, wherein the control function, comprises: 

IS a camera selection control that allows a user to select a camera angle from which is 

provided a video signal; and 

amultiple screen function thatprovides for display of video signals JBrommore than one 
camera angle. 

34. The electronic book of claim 33, wherein the video signals are tiled for display. 

20 35. The electronic book of claim 33, viierein the a first video signal is provided in a 
picture-in-picture format with a second video signal. 
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36. The electronic book of claim 28, wherein the electronic book is stored on a device 
having amemory, and wherein one or more of the streaming video, audio and text data are 
stored in the memory. 

37. . The electronic book of claim 28, A^erein the one or more of the plurality of streaming 
video, audio and text data are provided live with display of the link m the electronic 
book. 



38. The electronic book of claim 28, wherein the connection to the Intemet web site is 
completed using a wired communication system. 

39. The electronic book of claim 28, wherein the coimection to the bitemet web site is 
1 0 completed using a wireless communication system. 

40. The electronic book of claim28, wherein the connection to the Intemet web site uses 
an electronic link. 



41 . A compute-readable medium for controlling aconnecticnibetween an electronic book 
viewer and a world watch live site, comprising: 

1 S aninter&ce module that receives user inputs, the inputs comprising commands to select 

a viewing segment; 

amemoiy module coiq)led to the inter&cemodulefliat directs storage of the received 

inputs; 

a transmission module coipled to the memory mo dule that transmits the stored inputs 
20 to a remote location; and 
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a receiving module that receives video and audio data from the selected viewing 
segment. 

42 . The computer-readable medium of claim 4 1 , wherein the world watch live site is an 
bitemet web site. 

5 . 43. The computer-readable medium of claim 41, wherein die commands comprise: 

a camera pan command; 

a camera select command; 

a camera off command; 

a zoom command; 
10 a cam^a on command; and 

a camera tilt command. 

44. The computer-readable medium of claim 44, further comprising a display module 
coupled to the receiving module that displays the selected viewing segment. 

45. The compute-readable medium of claim44, \^ereinthereceivingmodulereceives 
1 5 multiple selected viewing segments and the display module displays the received multiple 

viewing segments simultaneously. 

46 . ITie conq)utCT-readable medium of claim 44, wherein the viewing segments conq)rise 
different views of the world watch live site. 

47. The computer-readable medium of claim 41 , wherein one or more of the viewing 
20 segments conqxrises a wide angle segmeaxt, the computer-readable medium iur&er conq)iisiDg 

a video conection module coupled to the display module that removes distortion fiom the wide 
angle segment before display. 
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48. The computer-readablemediumofclaiin41, wherein the video andaudio dataare 
saved in a memory. 

49. An electronic book, comprising: 
a data file; 

. agr^hicsfile; 
interactive files; 

electronic links included in tiie data file and the graphics file, wherein the electronic links 
couple the electronic book to the interactive files, the interactive files, including: 
an input file to receive conunands and data, and 
an ou^ut file to provide prompts and instmctions and to provide the inpat file; 

and 

acontrol program that operates to couple the electronic book to anint^active file upon 
activation of an electronic link. 

50. The electronic book of claim 49, wherein the interactive file is displayed on an 
1 5 electronic book viewer. 

5 1 . The electronic book of claim 49, wherein the interactive file is a world watch live site. 

52. The electronic book of claim 49, finther comprising an interactive file menu, the 
interactive file msnux including alinkto the mteractive file, wherein the menu includes alist of 
available interactive files, and^^iereinactivationofftelinkcomiectstfaememitotheii]^^ 

20 file, the interactive file being displayed on a screen of an electronic book viewer. 



5 . 
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